Nivolumab After Combined Modality Therapy in Treating Patients With High Risk Stage II-IIIB Anal Cancer

Inclusion Criteria: * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Age \>= 18 years * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients must have histologically proven stage IIB (T3N0M0 only), IIIA (T2N1M0), IIIB (T4N0M0), or IIIC (T3N1M0, T4N1M0) invasive squamous cell carcinoma of the anus or anorectum, according to the American Joint Committee on Cancer (AJCC) 8th edition; this may include tumors of non-keratinizing histology such as basaloid, transitional cell, or cloacogenic histology; individuals with squamous cell carcinoma of the anal margin are eligible if there is evidence of extension of the primary tumor into the anal canal * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients must have Eastern Cooperative Oncology Group (ECOG) performance status of 0-2 * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients must have hemoglobin levels of \> 9 g/dL (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patient must have a platelet count of \> 100,000/mm\^3 (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patient's absolute neutrophil count (ANC) level must be \> 1500/mm\^3 (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Serum creatinine must be =\< 1.5 X upper limit of normal (ULN) (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Total bilirubin must be \< 2 X ULN (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) =\< 2.5 X institutional upper limit of normal (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Albumin \>= 3.0 g/dL (within 2 weeks prior to registration) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients known to be human immunodeficiency virus (HIV)+ are permitted; patients with CD4 \> 200 and serum HIV viral load of \< 200 copies/mm\^3 are eligible, and in addition: * Participants must be purified protein derivative (PPD) negative; alternatively, the QuantiFERON-tuberculosis (TB) Gold In-Tube (QFT-GIT) assay (Cellestis Limited, Carnegie, Australia) can be used; an individual is considered positive for M. tuberculosis infection if the IFN-gamma response to TB antigens is above the test cut-off (after subtracting the background IFN-gamma response in the negative control); the result must be obtained within 20 weeks prior to enrollment; PPD positive (or Quantiferon assay positive) participants are permitted if prophylaxis has been completed prior to enrollment * No history of acquired immune deficiency syndrome (AIDS)-related complications within past year other than a history of low CD4+ T-cell count \> 200/mm\^3 prior to initiation of combination antiretroviral therapy; on study CD4+ T-cell count may not be informative due to chemoradiotherapy and should not be used as an exclusion criterion if low * Patient must be healthy on the basis of HIV disease with high likelihood of near normal life span were it not for the anal cancer * Participants MUST receive appropriate care and treatment for HIV infection, including antiretroviral medications when clinically indicated, and should be under the care of a physician experienced in HIV management; participants will be eligible regardless of antiretroviral medication (including no antiretroviral medication) provided there is no intention to initiate therapy or the regimen has been stable for at least 4 weeks with no intention to change the regimen within 12 weeks following enrollment * Patient must have =\< grade 2 diarrhea (participants with grade 1 diarrhea are eligible provided stool for ova/parasites and stool cryptosporidium studies are negative; * NOTE: HIV testing is not required for eligibility * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: For patients registering prior to start of chemoradiotherapy, baseline scans must have been completed within 4 weeks prior to registration * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Women of child bearing potential and sexually active males must use accepted and effective method(s) of contraception and/or abstain from sexual intercourse while on protocol treatment and for at least 5 months after the last dose of nivolumab (for female patients) and for at least 7 months after the last dose of nivolumab (for male patients) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Any surgery must have been completed \>= 4 weeks prior to starting study treatment * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: No uncontrolled intercurrent illness including, but not limited to ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: No prior treatment with an immune checkpoint inhibitor (anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CTLA4 monoclonal antibody) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: No patients with immunodeficiency or receiving systemic steroid therapy equivalent to \> 10 mg prednisone per day or any other form of immunosuppressive therapy within 7 days prior to Step 1 registration; topical corticosteroid or occasional inhaled corticosteroids are allowed * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: No live vaccines within 30 days prior to registration; examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, chicken pox, yellow fever, rabies, BCG, and typhoid (oral) vaccine; seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., Flu-Mist) are live attenuated vaccines and are not allowed * NOTE: no live vaccines may be administered while participating in the trial * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Previously irradiated patients (Arm S) must have received radiation per National Comprehensive Cancer Network guidelines; radiation therapy delivered on protocol (Arm T) will be reviewed * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients will be registered within 63 days following completion of standard chemoradiation for anal cancer; standard chemoradiation therapy is as defined * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must have histologically proven stage IIB (T3N0M0 only), IIIA (T2N1M0), IIIB (T4N0M0), or IIIC (T3N1M0, T4N1M0) invasive squamous cell carcinoma of the anus or anorectum, according to the AJCC 8th edition; this may include tumors of non-keratinizing histology such as basaloid, transitional cell, or cloacogenic histology; individuals with squamous cell carcinoma of the anal margin are eligible if there is evidence of extension of the primary tumor into the anal canal * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must have received at least 54 gray (Gy) of radiation to the PTVp (primary) and 45 Gy to PTVn (elective nodal region) for the treatment of the anal cancer * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must have ECOG performance status of 0-2 * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must have hemoglobin levels of \> 10 g/dL (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patient must have a platelet count of \> 100,000/mm\^3 (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patient's ANC level must be \> 1500/mm\^3 (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Serum creatinine must be =\< 1.5 X ULN (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Total bilirubin must be \< 2 X ULN (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: AST (SGOT)/ALT (SGPT) =\< 2.5 X institutional upper limit of normal (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Albumin \>= 3.0 g/dL (within 2 weeks prior to registration) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients known to be human immunodeficiency virus (HIV)+ patients with CD4 \> 200 and serum HIV viral load of \< 200 copies/mm\^3 are eligible; in addition: * Participants must be PPD negative; alternatively, the QuantiFERON-TB Gold In-Tube (QFT-GIT) assay (Cellestis Limited, Carnegie, Australia) can be used; an individual is considered positive for M. tuberculosis infection if the IFN-gamma response to TB antigens is above the test cut-off (after subtracting the background IFN-gamma response in the negative control); the result must be obtained within 20 weeks prior to enrollment; PPD positive (or Quantiferon assay positive) participants are permitted if prophylaxis has been completed prior to enrollment; NOTE: If patient completed chemoradiation on Step 1, PPD testing does not need to be performed again * No history of AIDS-related complications within past year other than a history of low CD4+ T-cell count \> 200/mm\^3 prior to initiation of combination antiretroviral therapy; on study CD4+ T-cell count may not be informative due to chemoradiotherapy should not be used as an exclusion criterion if low * Patient must be healthy on the basis of HIV disease with high likelihood of near normal life span were it not for the anal cancer * Participants MUST receive appropriate care and treatment for HIV infection, including antiretroviral medications when clinically indicated, and should be under the care of a physician experienced in HIV management; participants will be eligible regardless of antiretroviral medication (including no antiretroviral medication) provided there is no intention to initiate therapy or the regimen has been stable for at least 4 weeks with no intention to change the regimen within 12 weeks following enrollment * Patient must have =\< grade 2 diarrhea (participants with grade 1 diarrhea are eligible provided stool for ova/parasites and stool cryptosporidium studies are negative) * NOTE: HIV testing is not required for eligibility * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Scans done within 4 weeks of randomization to Step 2 * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patient must have recovered from all toxicities associated with chemoradiotherapy for anal cancer, to grade =\< 1 with the exception of alopecia * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Women of child bearing potential and sexually active males must use accepted and effective method(s) of contraception and/or abstain from sexual intercourse while on protocol treatment and for at least 5 months after the last dose of nivolumab (for female patients) and for at least 7 months after the last dose of nivolumab (for male patients) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: No uncontrolled intercurrent illness including, but not limited to ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: No prior treatment with an immune checkpoint inhibitor (anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CTLA4 monoclonal antibody) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: No patients with immunodeficiency or receiving systemic steroid therapy equivalent to \> 10 mg prednisone per day or any other form of immunosuppressive therapy within 7 days prior to the first dose of study medication; topical corticosteroid or occasional inhaled corticosteroids are allowed * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: No live vaccines within 30 days prior to the first dose of trial treatment and while participating in the trial; examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, chicken pox, yellow fever, rabies, BCG, and typhoid (oral) vaccine; seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., Flu-Mist) are live attenuated vaccines and are not allowed Exclusion Criteria: * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: For patients registering to Arm T, patients must not have received prior chemoradiotherapy for anal cancer * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients with an allogenic bone marrow/stem, cell or solid organ transplant are excluded * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Women MUST NOT be pregnant or breast-feeding due to the potential teratogenic harm or abortifacient effects to an unborn fetus and possible risk for adverse events in nursing infants with the treatment regimens being used; all patients must also not expect to conceive or father children from study registration and throughout their time on study treatment; for female patients this must continue until at least 5 months after the last dose of nivolumab and for male patients until at least 7 months after the last dose of nivolumab; all females of child bearing potential must have a serum or urine pregnancy test to rule out pregnancy within 2 weeks prior to registration; a female of childbearing potential is any woman, regardless of sexual orientation or whether they have undergone tubal ligation, who meets the following criteria: 1) has achieved menarche at some point, 2) has not undergone a hysterectomy or bilateral oophorectomy, or 3) has not been naturally postmenopausal (amenorrhea following cancer therapy does not rule out childbearing potential) for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients will be excluded if they have a T1 or M1, and T2N0 cancer * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients must not have had prior potentially curative surgery (abdominal, peritoneal resection) for carcinoma of the anus * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Participants may not be receiving any other standard anti-cancer therapy or experimental agent concurrently with the study drugs * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Individuals with a history of a different malignancy are ineligible except if they have been disease-free for at least 2 years and are deemed by the investigator to be at low risk for recurrence; individuals with the following cancers are eligible if diagnosed and treated within the past 5 years: cervical cancer in situ, and basal cell or squamous cell carcinoma of the skin * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patient must not have active autoimmune disease in the past 2 years * NOTE: This does not include patients with autoimmune disease controlled by medication, such as hypothyroidism; this eligibility includes only patients with endocrine disease controlled by hormone replacement, e.g. hypothyroid, adrenal insufficiency, or diabetes mellitus (DM) * REGISTRATION TO STEP 1 ELIGIBILITY CRITERIA: Patients must not have known interstitial lung disease that is symptomatic or may interfere with the detection or management of suspected drug-related pulmonary toxicity * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients with an allogenic bone marrow/stem, cell or solid organ transplant are excluded * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Women MUST NOT be pregnant or breast-feeding due to the potential teratogenic harm or abortifacient effects to an unborn fetus and possible risk for adverse events in nursing infants with the treatment regimens being used; all patients must also not expect to conceive or father children from study registration and throughout their time on study treatment; for female patients this must continue until at least 5 months after the last dose of nivolumab and for male patients until at least 7 months after the last dose of nivolumab; all females of child bearing potential must have a serum or urine pregnancy test to rule out pregnancy within 2 weeks prior to registration; a female of childbearing potential is any woman, regardless of sexual orientation or whether they have undergone tubal ligation, who meets the following criteria: 1) has achieved menarche at some point, 2) has not undergone a hysterectomy or bilateral oophorectomy, or 3) has not been naturally postmenopausal (amenorrhea following cancer therapy does not rule out childbearing potential) for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months) * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must not have had prior potentially curative surgery (abdominal, peritoneal resection) for carcinoma of the anus * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Participants may not be receiving any other standard anti-cancer therapy or experimental agent concurrently with the study drugs * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Individuals with a history of a different malignancy are ineligible except if they have been disease-free for at least 2 years and are deemed by the investigator to be at low risk for recurrence; individuals with the following cancers are eligible if diagnosed and treated within the past 5 years: cervical cancer in situ, and basal cell or squamous cell carcinoma of the skin * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patient must not have active autoimmune disease that has required systemic treatment in past 2 years * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must not have known interstitial lung disease that is symptomatic or may interfere with the detection or management of suspected drug-related pulmonary toxicity * REGISTRATION TO STEP 2 ELIGIBILITY CRITERIA: Patients must not have a history of allergic reactions attributed to compounds of similar chemical or biologic composition to nivolumab