Adjuvant Nivolumab Following Chemo-Radiation in Localized Muscle-Invasive Bladder Cancer

Inclusion Criteria: * Patients with muscularis propria invasion clinical stages 2 to 4 (T2-4b, N0 or N+, M0 or T1 with N+), who are not candidates for radical cystectomy. * Patients may have undergone partial cystectomy for removal of bladder tumor prior to chemoradiation. Patients who have M1 disease at any time prior to start of treatment are not eligible. * Staging is determined prior to chemoradiation * Patients have been evaluated by a urologic oncologist to determine eligibility for radical cystectomy prior to chemoradiation. Patients may not be candidates for radical cystectomy due to one or more reasons such as but not limited to comorbidities, age, surgical risk or patient refusal to undergo radical cystectomy. Patients who refuse to undergo radical cystectomy are not required to be evaluated by a urologic oncologist. * Patients must have histologically proven primary adenocarcinoma, transitional, squamous-cell, or sarcomatoid carcinoma primary of the bladder, urethra, or lower ureter. * Treating investigator has determined that the patients are not a candidate for radical cystectomy. Patients have been evaluated by a urologic oncologist to determine eligibility for radical cystectomy prior to chemoradiation. Patients may not be candidates for radical cystectomy due to one or more reasons such as, but not limited to, comorbidities, age, surgical risk, or patient refusal to undergo radical cystectomy. * Tumor tissue from the most recently resected site of disease (preferable) or from the transurethral resection that yielded the initial muscle invasive diagnosis must be provided for biomarker correlative analyses. Enrollment is permitted if adequate archived tissue is unavailable. * Patients must have received systemic radiosensitizing chemotherapy with definitive pelvic radiation therapy. Patients may have received partial amount of chemotherapy and radiation (both) to be eligible. * Platinum based chemotherapy prior to chemoradiation is permitted but not mandatory * Eastern Cooperative Oncology Group (ECOG) Performance Status of ≤2 * Age ≥18. * Adequate bone marrow function White Blood Cell (WBC) \> 2000/µl, neutrophils \>1500/µl, Hemoglobin \>9.0 g/dl. * Serum bilirubin and aminotransferase values less than 1.5 times the upper limit of the normal range * Creatinine clearance of 20 ml/min or greater as measured by the Cockroft-Gault formula * Able to start study treatment within 90 days of completion of chemoradiation. * Female subject of childbearing potential should have a negative urine or serum pregnancy within 72 hours prior to receiving the first dose of study medication. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required. * Female subjects of childbearing potential should be willing to use 2 methods of birth control or be surgically sterile, or abstain from heterosexual activity for the course of the study through 5 months after the last dose of study medication. Subjects of childbearing potential are those who have not been surgically sterilized or have not been free from menses for \> 1 year. * Male subjects should agree to use an adequate method of contraception starting with the first dose of study therapy through 7 months after the last dose of study therapy. * All toxicities attributed to prior anti-cancer therapy other than nephropathy, neuropathy, hearing loss, alopecia and fatigue must have resolved to Grade 1 (NCI CTCAE version 4.03) or baseline before administration of study drug. Subjects with toxicities attributed to prior anti-cancer therapy which are not expected to resolve and result in long lasting sequelae, such as neuropathy after platinum based therapy, are permitted to enroll. * Able to provide informed consent and willing to sign an approved consent form that conforms to federal and institutional guidelines. Exclusion Criteria: * Evidence of distant metastases or lymph node metastasis (es) that was not within the radiation field. * Known additional malignancy that is progressing or requires active treatment. Exceptions include basal cell carcinoma of the skin, squamous cell carcinoma of the skin. A history of localized early stage malignancy that has undergone potentially curative therapy or is low grade and does not require active treatment is allowed. * Diffuse bladder carcinoma in situ (CIS) that was not able to be encompassed in a boost radiotherapy volume * Patients with inflammatory bowel disease * Patients with active, known or suspected autoimmune disease. Subjects with vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger are permitted to enroll. * Patients with a condition requiring systemic treatment with either corticosteroids (\>10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 day of study drug administration. Inhaled, ocular, intraarticular, intranasal and topical steroids are permitted. * Patients with a known chronic immunocompromised state, HIV infection or active Hepatitis B or Hepatitis C infection. * Pregnancy or women of childbearing potential not willing to use contraception and men who are sexually active and not willing/able to use medically acceptable forms of contraception and breast-feeding women not willing to stop breastfeeding during study. * Severe active co-morbidity as determined by the investigator or principal investigator * Life expectancy less than 2 years