Testing Combination Erdafitinib and Enfortumab Vedotin in Metastatic Bladder Cancer After Treatment With Chemotherapy and Immunotherapy

Inclusion Criteria: * Patients must have histologically or cytologically documented locally advanced (T4b, any N; or any T, N 2-3) or metastatic (M1, Stage IV; or metastatic recurrence after locoregional treatment) urothelial carcinoma (including renal pelvis, ureters, urinary bladder, urethra). Patients with mixed histologies are required to have a dominant transitional cell pattern * Patients who had disease progression during or following treatment with at least one platinum-containing regimen (e.g., gemcitabine and cisplatin \[GC\], methotrexate, vinblastine, doxorubicin and cisplatin \[MVAC\], carboplatin and gemcitabine \[Carbo-Gem\]) and an immune checkpoint inhibitor (PD-1/ PD-L1 inhibitor including but not limited to: atezolizumab, pembrolizumab, durvalumab, avelumab, and nivolumab) * Received a first-line platinum-containing regimen in the metastatic setting or for inoperable locally advanced disease * Or received neo/adjuvant platinum-containing therapy for localized muscle-invasive UC, with recurrence/progression =\< 12 months following completion of therapy * Patients who received immune checkpoint inhibitor therapy in the neoadjuvant/adjuvant setting and had recurrent or progressive disease either during therapy or within 12 months of therapy completion are eligible. This criterion does not apply if the checkpoint inhibitor is contraindicated * Patients with metastatic urothelial carcinoma who are cisplatin-ineligible and progressed on upfront immune checkpoint inhibitor; or ineligible/refused immune checkpoint inhibitor therapy will be eligible for this trial * Patient who received prior antibody drug conjugate such as sacituzumab govitecan are allowed * Patients must have measurable disease, as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1. Previously irradiated lesions cannot be counted as target lesions unless there has been demonstrated progression in the lesion since radiotherapy and no other lesions are available for selection as target lesions * Patients must have FGFR2/3 activating alterations identified by tumor tissue or plasma ctDNA profiling using a Clinical Laboratory Improvement Act (CLIA) certified College of American Pathologists (CAP) accredited platform * Age \>= 18 years, for ability to comply with protocol * Because no dosing or adverse event data are currently available on the use of erdafitinib in combination with enfortumab vedotin in patients \< 18 years of age, children are excluded from this study * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 (Karnofsky \>= 60%) * Absolute neutrophil count \>= 1,500/mcL (within 14 days prior to beginning trial treatment) * Platelets \>= 100,000/mcL (within 14 days prior to beginning trial treatment) * Hemoglobin \>= 9 g/dL (within 14 days prior to beginning trial treatment) * Measured or calculated creatine clearance (CrCl) \>= 30 ml/min (glomerular filtration rate \[GFR\] can also be used in place of creatinine CrCl) (within 14 days prior to beginning trial treatment) * Total bilirubin =\< 1.5 x ULN (institutional upper limit of normal) OR direct bilirubin =\< ULN for subjects with total bilirubin levels \> 1.5 x ULN (within 14 days prior to beginning trial treatment) * Aspartate aminotransferase (AST)(serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT)(serum glutamic pyruvic transaminase \[SGPT\]) =\< 2.5 x institutional ULN (=\< 5 x ULN for subjects with liver metastasis) (within 14 days prior to beginning trial treatment) * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression (CNS metastases have been clinically stable for at least 4 weeks prior to screening and baseline scans show no evidence of new or enlarged metastasis) * Patients with a history of prostate cancer (T2NXMX or lower with Gleason score =\< 7) treated with definitive intent (surgically or with radiation therapy) at least 1 year prior to study entry are eligible, provided that the subject is considered prostate cancer-free and the following criteria are met: * Patients who have undergone radical prostatectomy must have undetectable prostate specific antigen (PSA) for \> 1 year and at screening * Patients who have had radiation must have a PSA doubling time \> 1 year (based on at least 3 values determined \>1 month apart) and a total PSA value that does not meet Phoenix criteria for biochemical recurrence (i.e., \< 2.0 ng/mL above nadir) * Patients with untreated low-risk prostate cancer (Gleason score =\< 6) on active surveillance with PSA doubling time \>1 year (based on at least 3 values determined \> 1 month apart) are also eligible * Patients who have undergone an ophthalmologic examination and have no active eye disease which would be likely to increase the risk of eye toxicity * The effects of erdafitinib and enfortumab vedotin on the developing human fetus are unknown. For this reason and because FGFR inhibitors and humanized antibody-drug conjugate (ADC) agents are known to be teratogenic, women of child-bearing potential must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and 3 months after completion of erdafitinib and enfortumab vedotin administration. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 5 months after completion of erdafitinib and enfortumab vedotin administration * Ability to understand and willingness to sign a written informed consent document Exclusion Criteria: * Patients who have had chemotherapy, targeted therapies, immunotherapy, or treatment with an investigational anticancer agent within 2 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study * Patients who have not recovered from adverse events due to prior anti-cancer therapy (including ongoing sensory or motor neuropathy of grade 2 or higher) (i.e., have residual toxicities \> grade 1 or returned to baseline) with the exception of alopecia. Subjects with =\< grade 2 immunotherapy- related hypothyroidism or panhypopituitarism may be enrolled when well-maintained/controlled on a stable dose of hormone replacement therapy (if indicated). Subjects with ongoing \>= grade 3 immunotherapy-related hypothyroidism or panhypopituitarism are excluded. Subjects with ongoing immunotherapy related colitis, uveitis, myocarditis, or pneumonitis or subjects with other immunotherapy related adverse events (AEs) requiring high doses of steroids (\> 20 mg/day of prednisone or equivalent) are excluded * Patients who have previously received enfortumab vedotin or other MMAE-based ADCs * Patients who have had prior treatment with an FGFR inhibitor * History of allergic reactions attributed to compounds of similar chemical or biologic composition to erdafitinib and enfortumab vedotin * Patients receiving any medications or substances that are strong inhibitors or inducers of CYP3A are ineligible. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Patients with a history of any corneal or retinal abnormality likely to increase the risk of eye toxicity * Patients with uncontrolled intercurrent illness and currently receiving systemic antimicrobial treatment for active infection (viral, bacterial, or fungal) at the time of starting treatment. Routine antimicrobial prophylaxis is permitted * Patients with psychiatric illness/social situations that would limit compliance with study requirements * Subjects who have received radiotherapy within 2 weeks prior to start of treatment. Subject must have recovered adequately from the toxicity from the intervention prior to starting study treatment * Patients with uncontrolled diabetes. Uncontrolled diabetes is defined as hemoglobin A1c (HbA1c) \>= 8% or HbA1c 7% to \< 8% with associated diabetes symptoms (polyuria or polydipsia) that are not otherwise explained * Subjects who have received major surgery within 4 weeks prior to start of treatment. Subject must have recovered adequately from complications from the intervention prior to starting study treatment * Subjects who have received a prior allogeneic stem cell or solid organ transplant * Has persistent phosphate level \> ULN during screening (within 14 days of treatment and prior to cycle 1 day 1) and despite medical management * Has a history of or current uncontrolled cardiovascular disease including: * Unstable angina, myocardial infarction, or known congestive heart failure class II-IV within the preceding 12 months; cerebrovascular accident or transient ischemic attack within the preceding 3 months * Any of the following: sustained ventricular tachycardia, ventricular fibrillation, Torsades de Pointes, cardiac arrest, Mobitz II second degree heart block or third degree heart block; known presence of dilated, hypertrophic, or restrictive cardiomyopathy * Corrected QT interval (QTc) prolongation as confirmed by triplicate assessment at screening (Fridericia;QTc \> 480 milliseconds) * Subjects with a history of another invasive malignancy within 3 years before the first dose of study drug, or any evidence of residual disease from a previously diagnosed malignancy. Subjects with nonmelanoma skin cancer or carcinoma in situ of any type (if complete resection was performed) are allowed