Brentuximab Vedotin and Nivolumab in Treating Patients With Early Stage Classic Hodgkin Lymphoma

Inclusion Criteria: * Documented informed consent of the participant and/or legally authorized representative. * Assent, when appropriate, will be obtained per institutional guidelines. * Eastern Cooperative Oncology Group (ECOG) =\< 2. * Histologically confirmed diagnosis of classical Hodgkin lymphoma (cHL) by current World Health Organization classification (nodular sclerosis, mixed cellularity, lymphocyte rich, lymphocyte depleted, or classical Hodgkin lymphoma, NOS \[not otherwise specified\]) at local enrolling center. Nodular lymphocyte-predominant Hodgkin lymphoma is excluded * Stage IA, IB, IIA, or IIB cHL by Cotswold modified Ann Arbor staging done prior to any treatment with ABVD. * Must have prior to standard of care ABVD treatment at least one lesion that is \> 1.5 cm in the longest diameter on cross-sectional imaging and measureable in two perpendicular dimensions on CT and fludeoxyglucose (FDG) avid by PET. Exclusion Criteria: * Patients must be naive in terms of any prior therapy for Hodgkin lymphoma (including immunotherapy, chemotherapy or radiation therapy) with the exception that they may have received up to 2 cycles of ABVD as standard of care therapy prior to enrollment, as long as they can start protocol therapy (therapy administered in Arms A, B1/B2, or C) within timelines specified by the trial. * Peripheral sensory neuropathy \> grade 1 or any peripheral motor neuropathy. * History of another primary malignancy that has not been in remission for at least 3 years. (The following are exempt from the 3-year limit: nonmelanoma skin cancer, fully excised melanoma in situ \[Stage 0\], curatively treated localized prostate cancer, and cervical carcinoma in situ on biopsy or a squamous intraepithelial lesion on Papanicolau \[PAP\] smear) * Known cerebral/meningeal disease. * History of progressive multifocal leukoencephalopathy (PML). * Known history of pancreatitis. * Documented history of a cerebral vascular event (stroke or transient ischemic attack), unstable angina, myocardial infarction, or cardiac symptoms consistent with New York Heart Association class III-IV within 6 months prior to enrollment * Uncontrolled cardiac disease including ventricular dysfunction, left ventricular ejection fraction \< 45%, coronary artery disease, or arrhythmias. * Known hypersensitivity to recombinant proteins, murine proteins, or to any excipient contained in the drug formulation of brentuximab vedotin, nivolumab, or any component of ABVD. * Known active infection with hepatitis B or hepatitis C. Patients who are hepatitis B carriers can enroll if have a negative hepatitis B polymerase chain reaction (PCR) DNA test and are on hepatitis B suppressive medication management with entecavir or lamivudine. Patients with past active hepatitis C virus (HCV) infection are eligible if they are PCR negative after curative therapy. Testing to be done only in patients suspected of having infections or exposures. * Known active infection with human immunodeficiency virus (HIV). Patients who are HIV positive can enroll if CD4 count is \> 200/uL and have an undetectable or unquantifiable HIV viral load within 28 days of enrollment, have concurrent management with infectious disease specialists, and are on stable combination antiretroviral therapy. Participants are required to be on antiretroviral regimens that are in accordance with the current International acquired immune deficiency syndrome (AIDS) Society guidelines concurrently with chemotherapy. The specific agents are at the discretion of the Investigator and the use of investigational agents currently available on an expanded access basis is allowed. Use of experimental antiretroviral agents or those containing zidovudine (including Combivir and Trizivir) or ritonavir (includes Norvir or Kaletra), Cobicistat, Didanosine (Videx or Videx EC), or similar potent CYP3 inhibitors are prohibited. In order to be eligible, participants taking zidovudine or ritonavir, Cobicistat, Didanosine, or other CYP3 inhibitors must change to a different regimen 7 days prior to therapy initiation. Changes to highly active antiretroviral therapy (HAART) therapy during the study may be made if medically necessary (toxicity, failure of regimen, etc.). Participants must be on HAART for at least 7 days prior to therapy. HIV testing to be done only in patients suspected of having infections or exposures * Subjects with active interstitial pneumonitis. * Subjects with active, known or suspected autoimmune disease. Subjects with vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger are permitted to enroll. * Females only: pregnant or breastfeeding. * Any other condition that would, in the investigator's judgment, contraindicate the patient's participation in the clinical study due to safety concerns with clinical study procedures. * Prospective participants who, in the opinion of the investigator, may not be able to comply with all study procedures (including compliance issues related to feasibility/logistics).