Intensity-Modulated Radiation Therapy & Nivolumab for Recurrent or Second Primary Head & Neck Squamous Cell Cancer

Inclusion Criteria: * Patients with recurrent squamous cell carcinoma or a second primary arising in a previously irradiated field * Life expectancy of greater than 6 months * Patients cannot have distant metastases and have to be candidates for curative re-irradiation * Patients with salivary gland tumors are excluded (patients with nasopharynx or sinonasal cancers can participate) * Patients with unresectable disease are eligible * Patients who undergo surgical resection will be allowed regardless of human papilloma virus (HPV) status provided they have one of the following criteria: * Positive margins on pathology * Evidence of extracapsular spread on nodal pathology * Gross residual disease on postoperative or simulation imaging * N2/3 disease * T3/4 disease * Multifocal perineural invasion and/or lymphovascular space invasion * The majority of the anticipated target volume (\> 50%) must have been previously treated to ≥ 40 Gy; prior radiation therapy (RT) must have been completed \> 6 months prior to initiation of IMRT reirradiation; if previous RT records are unavailable, investigators can estimate the dose to previously treated tissues based on completion notes or other treatment history * An Eastern Cooperative Oncology Group (ECOG) performance score 0-2 * Granulocytes \> 1500/mm³ * Platelets \> 100,000/mm³ * Bilirubin \< 1.5 mg/dl * Creatinine \< 1.5 mg/dl * No other concurrent invasive malignancies treated for the past year (localized prostate cancer or early stage skin cancer are not exclusion criteria) * Patients with carotid artery involvement or encasement will be allowed provided they have no symptoms related to carotid involvement * No prior exposure to immunotherapy agents * Ability to understand and the willingness to sign a written informed consent document Exclusion Criteria: * Any known factors that would pose a contraindication to receiving nivolumab * Recursive partitioning analysis (RPA) class III patients defined as those expected to begin reirradiation within 2 years of first course of radiation therapy AND are percutaneous endoscopic gastrostomy (PEG) dependent or have a tracheostomy (patients who have undergone total laryngectomy are not excluded) * Patients with metastases * Prior treatment with a programmed cell death protein-1 (PD-1)/programmed death-ligand 1 (PD-L1) inhibitor * Female patients of childbearing potential must have a negative serum pregnancy test within 7 days prior to enrollment * Patients with primary salivary gland cancers are excluded * Patients who have had chemotherapy or biological therapy within 4 weeks of registration * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, autoimmune disease requiring systemic steroids, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Patients who are pregnant or breast-feeding * Patients with known active human immunodeficiency virus (HIV), hepatitis (hep) B, or hep C infection * Subjects with a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study drug administration; inhaled or topical steroids and adrenal replacement doses \> 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease