Durvalumab and Vicineum in Subjects With High-Grade Non-Muscle-Invasive Bladder Cancer Previously Treated With Bacillus Calmette-Guerin (BCG)

* INCLUSION CRITERIA: * Patients must have histologically or cytologically confirmed by National Cancer Institute (NCI) Laboratory of Pathology as high-grade non-muscle invasive urothelial (transitional cell carcinoma) of the bladder as follows: * Carcinoma-in-situ (CIS) with or without papillary tumors * High-grade Ta or T1 disease based on a biopsy/transurethral resection of bladder tumor (TURBT) performed within 12 weeks of the initial dose of study treatment. If multiple bladder biopsies/TURBTs are required to confirm eligibility, the timing of the last bladder biopsy to the initial dose of study treatment must be within 12 weeks. * Patients with persistent T1 high grade disease on TURBT following a single induction course of BCG (Bacillus Calmette-Guerin) (at least 5 of 6 doses) may also be eligible for this trial provided that the patient is surgically unfit for cystectomy as deemed by the investigator or the patient declines cystectomy. * Subjects with BCG unresponsive disease as defined by the Society of Urologic Oncology and the Food and Drug Administration (FDA): Subjects must have received at least two courses of intravesical BCG (at least 5 of 6 induction doses of BCG and at least 2 of 3 maintenance doses of BCG under a maintenance regimen or at least 2 doses of a repeat induction course). Please note exception above for persistent T1 disease. There is no upper limit on the amount of prior BCG a subject may have received. * Patients who have met eligibility criterion above must have received last BCG dose within a year of enrollment. * The investigator must document that he/she believes the subject would not benefit from additional BCG treatment at the time of study entry. * Age \>= 18 years at time of signing the informed consent form (ICF). Because no dosing or adverse event data are currently available on the use of Vicineum in combination with durvalumab in patients \<18 years of age, children are excluded from this study, but will be eligible for future pediatric trials. Furthermore, non-muscle invasive bladder cancer (NMIBC) does not occur in children. * Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1 * Adequate organ and marrow function as defined below: * Hemoglobin \>= 9.0 g/dL * Absolute neutrophil count (ANC) \>= 1.5 x 10\^9/L (\> 1500 per mm\^3) * Platelet count \>= 75 x 10\^9/L (\>75,000 per mm\^3) * Serum bilirubin less than or equal to 1.5 x institutional upper limit of normal (ULN). * Aspartate aminotransferase (AST) serum glutamic oxaloacetic transaminase (SGOT)/alanine aminotransferase (ALT) serum glutamate-pyruvate transaminase (SGPT) less than or equal to 2.5 x institutional ULN * Creatinine CL\>40 mL/min by the Cockcroft-Gault formula (Cockcroft and Gault 1976) or by 24-hour urine collection for determination of creatinine clearance: * Males: Creatinine CL (mL/min) = (Weight (kg) x (140 - Age))/72 x serum creatinine (mg/dL) * Females: Creatinine CL (mL/min) = (Weight (kg) x (140 - Age) x 0.85)/72 x serum creatinine (mg/dL) * Female subjects must either be of non-reproductive potential (i.e., post-menopausal as described below) OR history of surgical sterilization (hysterectomy, bilateral tubal ligation, or bilateral oophorectomy) or must have a negative serum pregnancy test upon study entry. * Women \<50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of exogenous hormonal treatments and if they have luteinizing hormone and follicle-stimulating hormone levels in the post-menopausal range for the institution * Women \>= 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of all exogenous hormonal treatments, had radiation-induced menopause with last menses \>1 year ago, or had chemotherapy-induced menopause with last menses \>1 year ago * The effects of Vicineum and durvalumab on the developing human fetus are unknown. For this reason, all sexually active subjects agree to use barrier contraception (i.e., condoms) while receiving study treatment and for 120 days following their last dose of study treatment. Female subjects of child-bearing potential and male subjects whose sexual partners are women of childbearing potential (WOCBP) agree to use barrier contraception and a second form of contraception while receiving study treatment and for 4 months following their last dose of study treatment. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. * Written informed consent obtained from the subject prior to performing any protocol- related procedures * Subject is willing and able to comply with the protocol for the duration of the study including undergoing treatment and scheduled visits and examinations including follow up. * Body weight \> 30 kg EXCLUSION CRITERIA: * Patients who are receiving any other investigational agents. * QT interval corrected for heart rate using Fridericia's formula the corrected QT interval by Fridericia (QTcF) \>=470 ms. (Any clinically significant abnormalities detected require triplicate ECG results and a mean QT interval corrected for heart rate using Fridericia's formula (QTcF) \>=470 ms calculated from 3 ECGs.) * History of allergic reactions attributed to compounds of similar chemical or biologic composition to Vicineum or durvalumab or other agents used in the study. * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements. Urinary tract infections (UTIs) are excluded from being an exclusion criterion for treatment unless they are grade 3 or higher. * Pregnant women are excluded from this study because it is unknown whether Vicineum and/or durvalumab have any teratogenic effects. In nursing mothers, breastfeeding should be discontinued as these medications may have the potential risk for adverse events in nursing infants secondary to treatment of the mother. * Any previous treatment with a programmed cell death 1 (PD-1) or programmed death-ligand 1 (PD-L1) inhibitor, including durvalumab * Evidence of non-bladder urothelial (transitional cell) carcinoma by biopsy, cytology, or radiological imaging within the past 2 years of treatment (e.g., upper tract transitional cell carcinoma, urethral urothelial carcinoma). * Subjects with hydronephrosis, except for those subjects where hydronephrosis has been longstanding (i.e., predates the diagnosis of the CIS (carcinoma in situ), Ta, or T1 by more than 2 years) and diagnostic evaluation at screening shows no evidence of tumor causing the hydronephrosis. * Any other anticancer therapy (e.g., chemotherapy, biologic therapy, immunotherapy, targeted therapy, endocrine therapy, radiation therapy, intravesical therapy, investigational agent) within 28 days of the first dose of study therapy (and within 6 weeks for nitrosourea or mitomycin C) other than a single dose of intravesical chemotherapy which is permitted between 28 days and 14 days prior to the first dose of study treatment. * The subject has a diagnosis of another malignancy within 2 years before the first dose of study treatment, except for superficial skin cancer, localized prostate cancer on active surveillance, or localized solid tumors deemed cured by surgery and not treated with systemic anticancer therapy and not expected to require anticancer therapy in the next 2 years i.e., while the subject may be taking study treatment. * Current or prior use of immunosuppressive medication within 28 days before the first dose of durvalumab, with the exceptions of intranasal and inhaled corticosteroids or systemic corticosteroids at physiological doses, which are not to exceed 10 mg/day of prednisone, or an equivalent corticosteroid. * Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease \[e.g., colitis or Crohn's disease\], diverticulitis \[with the exception of diverticulosis\], systemic lupus erythematosus, Sarcoidosis syndrome, or Wegener syndrome \[granulomatosis with polyangiitis, Grave's disease, rheumatoid arthritis, hypophysitis, uveitis, etc.\]). The following are exceptions to this criterion: * Subjects with vitiligo or alopecia * Subjects with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormonal replacement * Any chronic skin condition that does not require systemic therapy * Subjects without active disease in the last 5 years may be included but only after consultation with the Principal Investigator * Subjects with celiac disease controlled by diet alone * History of primary immunodeficiency. * History of allogeneic organ transplant. * History of hypersensitivity to durvalumab or any excipient * History of hypersensitivity to Vicineum or its components * Active infection with tuberculosis (clinical evaluation that includes clinical history, physical examination, and radiographic findings, and purified protein derivative (PPD) testing if indicated), hepatitis B (known positive HBV surface antigen (HBsAg) result, hepatitis C, or human immunodeficiency virus (positive HIV 1/2 antibodies). Patients with HIV are excluded from participating on this clinical trial because their immunodeficiency would confound the evaluation of adverse events which would hinder meeting the primary objective. Subjects with a past or resolved HBV infection (defined as the presence of hepatitis B core antibody \[anti-HBc\] and absence of HBsAg) are eligible. Subjects positive for hepatitis C (HCV) antibody are eligible only if polymerase chain reaction is negative for HCV ribonucleic acid (RNA). * History of leptomeningeal carcinomatosis * Receipt of live attenuated vaccination within 30 days prior to the first dose of Vicineum or durvalumab * Any condition that, in the opinion of the investigator, would interfere with evaluation of study treatment or interpretation of patient safety or study results * Subjects with uncontrolled seizures * Any unresolved toxicity National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) Grade \>=2 from previous anticancer therapy with the exception of alopecia, vitiligo, and the laboratory values defined in the inclusion criteria * Subjects with Grade \>=2 neuropathy will be evaluated on a case-by-case basis after consultation with the Principal Investigator. * Subjects with irreversible toxicity not reasonably expected to be exacerbated by treatment with durvalumab may be included only after consultation with the Principal Investigator.