Testing Anti-Cancer Drugs Erdafitinib With or Without Atezolizumab in Patients With Localized Bladder Cancer Not Able to Receive Cisplatin Chemotherapy, NERA Trial

Inclusion Criteria: * Patients must have muscle-invasive disease, tumor stage T2-T4, N0-1, based on transurethral resection of bladder tumor (TURBT) performed within 8 weeks prior to enrollment * Clinical stage T2-T4, N0 or N1, M0 by CT chest abdomen pelvis (or CT chest and MRI abdomen pelvis). Ultrasound, fluorine-18-fluorodeoxyglucose positron emission tomography (FDG-PET), and plain x-rays are not acceptable methods of evaluating clinical staging in the absence of CT or MRI scans * Histologically-confirmed urothelial carcinoma. Urothelial carcinoma with variant differentiation is allowed regardless of percentage. Other variant histology is allowed if urothelial carcinoma is the predominant histology (\>= 50%), except neuroendocrine (any neuroendocrine carcinoma would be an exclusion criteria) * Patients must be eligible and planned for curative-intent RC, as determined by a urologic oncologist * Patients who are ineligible for or decline cisplatin-based chemotherapy. * Cisplatin-ineligibility defined as \>= 1 of the following criteria (modified from the Galsky criteria): Eastern Cooperative Oncology Group (ECOG) performance status (PS) of \>= 2, either estimated or measured creatinine clearance (CrCl) or glomerular filtrate rate (GFR) \< 60 mL/min, Common Terminology Criteria for Adverse Events (CTCAE) version (v) 5 grade \>= 2 audiometric hearing loss or grade \>= 2 peripheral neuropathy * For cisplatin-eligible patients who decline cisplatin-based neoadjuvant chemotherapy (NAC), the subject's refusal for cisplatin must be clearly documented. Subjects must be informed that cisplatin-based NAC can improve cure rates, and it is unknown whether FGFR3/2 aberrant MIBC have better cure rates with neoadjuvant erdafitinib than cisplatin-based NAC * Patients with susceptible FGFR3/2 alterations, based on tumor tissue testing, or blood ctDNA testing, performed by the local institution. (Given presence of intratumoral heterogeneity in FGFR3 status, a sample of the deeper part of the invasive tumor is preferred for tissue FGFR screening using baseline TURBT sample) * FGFR3 gene mutations (R248C, S249C, G370C, Y373C) or FGFR gene fusions (FGFR3-TACC3, FGFR3-BAIAP2L1, FGFR2-BICC1, FGFR2-CASP7), as defined by the current Food and Drug Administration (FDA) indication for erdafitinib, determined by a laboratory certified by Clinical Laboratory improvement Amendments (CLIA). The principal investigator (PI) will review all clinical testing report to confirm eligibility * The FGFR screening assay is chosen by the local investigators depending on what is available (FoundationOne CDx, TSO500, Therascreen or others). If local FGFR screening capabilities are not available, study team may be able to provide funding support * FGFR status will be confirmed centrally and retrospectively using whole exome sequencing (WES) and ribonucleic acid (RNA) sequencing at the National Clinical Laboratory Network (NCLN) genomics lab, however patients can proceed with randomization and study treatment prior to confirmation testing results being made available * Patients with the following prior therapies are allowed: * PD-1/PD-1 immune checkpoint inhibitors (ICIs) for non-muscle invasive bladder cancer (NMIBC) is allowed if last dose was given \>1 year prior to randomization * Hormone-replacement therapy or oral contraceptives * Herbal therapy \>1 week prior to cycle 1, day 1 (herbal therapy intended as anti-cancer therapy must be discontinued at least 1 week prior to cycle 1, day 1) * Age \>= 18 years; because no dosing or adverse event data are currently available on the use of erdafitinib in combination with atezolizumab in patients \< 18 years of age, children are excluded from this study * ECOG performance status =\< 2 (Karnofsky \>= 60%) * Absolute neutrophil count \>= 1,500/mcL (without granulocyte colony stimulating factor support) (within 14 days of study registration) * Platelets \>= 100,000/mcL (within 14 days of study registration) * Hemoglobin \>= 9 g/dL (stable for 2 weeks without transfusion or erythropoiesis-stimulating agent) (within 14 days of study registration) * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (however, patients with known Gilbert disease who have serum bilirubin level \>= 3 x ULN may be enrolled) (within 14 days of study registration) * Aspartate aminotransferase (AST) (serum aspartate aminotransferase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) =\< 2.5 x institutional ULN or =\< 5 x ULN for patients with liver metastases (within 14 days of study registration) * Alkaline phosphatase =\< 2.5 x ULN (=\< 5 x ULN for patients with documented liver or bone metastases) (within 14 days of study registration) * Serum creatinine =\< 1.5 x institutional ULN (within 14 days of study registration) OR creatinine clearance \>= 30 mL/min by Cockcroft-Gault (within 14 days of study registration) * Glomerular filtration rate (GFR) \>= 30 mL/min/1.73 m\^2 given this is a cisplatin-ineligible patient population (within 14 days of study registration) * International normalized ratio (INR) and activated partial thromboplastin time (aPTT) =\< 1.5 x ULN (This applies only to patients who do not receive therapeutic anticoagulation; patients receiving therapeutic anticoagulation, such as low-weight heparin or warfarin, should be on a stable dose.) (within 14 days of study registration) * Serum albumin \>= 2.5 g/dL (within 14 days of study registration) * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * Patients should be New York Heart Association Functional Classification of class II or better * Patients should be greater than 4 weeks from receiving a live attenuated vaccine. Please note that non-live vaccines for seasonal influenza and vaccines intended to prevent SARS-CoV-2 and coronavirus disease 2019 (COVID-19) are allowed * Patients with nonmetastatic concomitant upper tract urothelial carcinoma (in addition to MIBC with susceptible FGFR alterations) are eligible for this trial * The effects of erdafitinib and atezolizumab on the developing human fetus are unknown. For this reason and because FGFR inhibitors and monoclonal antibodies are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, 1 month after completion of erdafitinib single agent, and 5 months after erdafitinib and atezolizumab combination. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately * Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity who have a legally-authorized representative (LAR) and/or family member available will also be eligible Exclusion Criteria: * History of another active malignancy within 2 years prior to screening, with the exception of malignancies with a negligible risk of metastasis or death (e.g., 5-year OS rate 90%), such as adequately treated carcinoma in situ of the cervix, ductal carcinoma in situ, stage I uterine cancer, non-melanoma skin cancer, resected in situ cancer, or definitively treated low risk prostate cancer * History of leptomeningeal disease * Patients with any metastases (including brain) will be excluded from this study as this study is enrolling patients with nonmetastatic urothelial bladder cancer * Uncontrolled tumor-related pain * Patients requiring pain medication must be on a stable regimen at study entry * Symptomatic lesions (e.g., bone metastases or metastases causing nerve impingement) amenable to palliative radiotherapy should be treated prior to enrollment. Patients should be recovered from the effects of radiation. There is no required minimum recovery period * Asymptomatic metastatic lesions that would likely cause functional deficits or intractable pain with further growth (e.g., epidural metastasis that is not currently associated with spinal cord compression) should be considered for loco-regional therapy if appropriate prior to enrollment * Major surgical procedure, other than for diagnosis, within 4 weeks prior to initiation of study treatment, or anticipation of need for a major surgical procedure during the study * Patients with the following prior therapies are not allowed: * Prior FGFR inhibitors or PD-1/PD-L1 immune check point inhibitors for MIBC. * Prior platinum chemotherapy * Patients with history of idiopathic pulmonary fibrosis, pneumonitis (including drug induced), organizing pneumonia (i.e., bronchiolitis obliterans, cryptogenic organizing pneumonia, etc.), or evidence of active pneumonitis on screening chest computed tomography (CT) scan. History of radiation pneumonitis in the radiation field (fibrosis) is permitted * Uncontrolled pleural effusion, pericardial effusion, or ascites requiring recurrent drainage procedures (once monthly or more frequently) * Patients with indwelling catheters (e.g., PleurX) are allowed * Patients with known primary immunodeficiency * Severe infection within 4 weeks prior to initiation of study treatment, including, but not limited to, hospitalization for complications of infection, bacteremia, or severe pneumonia, or any active infection that could impact patient safety * Treatment with therapeutic oral or IV antibiotics within 2 weeks prior to initiation of study treatment * Patients receiving prophylactic antibiotics (e.g., to prevent a urinary tract infection or chronic obstructive pulmonary disease exacerbation) are eligible for the study * Patients with prior allogeneic bone marrow transplantation or prior solid organ transplantation * History or risk of autoimmune disease, including, but not limited to, myasthenia gravis, myositis, autoimmune hepatitis, systemic lupus erythematosus, rheumatoid arthritis, inflammatory bowel disease, antiphospholipid antibody syndrome, Wegener granulomatosis, Sjogren syndrome, Guillain-Barre syndrome, or multiple sclerosis, with the following exceptions: * Patients with a history of autoimmune-related hypothyroidism on a stable dose of thyroid replacement hormone may be eligible * Patients with controlled Type 1 diabetes mellitus on a stable insulin regimen may be eligible * Patients with eczema, psoriasis, lichen simplex chronicus, or vitiligo with dermatologic manifestations only (e.g., patients with psoriatic arthritis would be excluded) are permitted provided all of the following conditions are met: * Rash must cover less than 10% of body surface area (BSA) * Disease is well controlled at baseline and only requiring low potency topical steroids. * No acute exacerbations of underlying condition within the last 12 months (not requiring psoralen plus ultraviolet A radiation \[PUVA\], methotrexate, retinoids, biologic agents, oral calcineurin inhibitors; high potency or oral steroids) within the previous 12 months * Treatment with systemic immunostimulatory agents (including, but not limited to, interferon \[IFN\]-alpha or interleukin \[IL\]-2) within 4 weeks or five half-lives of the drug (whichever is longer) prior to cycle 1, day 1 * Treatment with systemic immunosuppressive medications (including, but not limited to, prednisone, cyclophosphamide, azathioprine, methotrexate, thalidomide, and anti-tumor necrosis factor \[anti-TNF\] agents) within 2 weeks prior to cycle 1, day 1 or anticipation of need for systemic immunosuppressive medication during study treatment, with the following exceptions: * Patients who have received acute, low dose, systemic immunosuppressant medications or one-time pulse dose of systemic immunosuppressant medication (e.g., 48 hours of corticosteroids for a contrast allergy) are eligible after principal investigator confirmation has been obtained * Patients who have received mineralocorticoids (e.g., fludrocortisone), corticosteroids for chronic obstructive pulmonary disease (COPD) or asthma, or low-dose corticosteroids for orthostatic hypotension or adrenocortical insufficiency are eligible * Patients with significant cardiovascular disease (such as New York Heart Association class III or greater cardiac disease, myocardial infarction, or cerebrovascular accident) within 3 months prior to initiation of study treatment, unstable arrhythmia, or unstable angina * Patients with significant endocrine alterations of calcium/phosphate homeostasis, unless well controlled * Uncontrolled or symptomatic hypercalcemia (ionized calcium \> 1.5 mmol/L, calcium \> 12 mg/dL or corrected serum calcium \> ULN) * Persistent hyperphosphatemia \> ULN despite medical management, within 14 days of first dose of study drug * Patients with active ocular disorder or abnormality likely to significantly increase the risk of eye toxicity: * History of or current evidence of central serous retinopathy or retinal vascular occlusion * Active wet, age-related macular degeneration * Diabetic retinopathy with macular edema * Uncontrolled glaucoma (per local standard of care) * Corneal pathology such as keratitis, keratoconjunctivitis, keratopathy, corneal abrasion, and inflammation or ulceration * Any other disease, metabolic dysfunction, physical examination finding, or clinical laboratory finding that contraindicates the use of an investigational drug, may affect the interpretation of the results, or may render the patient at high risk from treatment complications * Patients with psychiatric illness/social situations that would limit compliance with study requirements * Patients who have not recovered from adverse events due to prior anti-cancer therapy (i.e., have residual toxicities \> grade 1) with the exception of alopecia * Treatment with any other agent administered for the treatment of the patient's cancer, within four half-lives or 4 weeks prior to cycle 1, day 1, whichever is shorter * History of allergic reactions attributed to compounds of similar chemical or biologic composition to erdafitinib or atezolizumab * History of severe allergic, anaphylactic, or other hypersensitivity reactions to chimeric antibodies, fusion proteins, or Chinese hamster ovary cell products or to any component of the atezolizumab formulation * Patients requiring any medications or substances that are moderate CYP2C9 inducers and strong CYP3A4 inducers are ineligible. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Pregnant women are excluded from this study because erdafitinib is an FGFR inhibitor and atezolizumab is a monoclonal antibody with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with erdafitinib and atezolizumab, breastfeeding should be discontinued if the mother is treated with erdafitinib and atezolizumab