Nivolumab and Brentuximab Vedotin in Treating Older Patients With Untreated Hodgkin Lymphoma

Inclusion Criteria: * Classical Hodgkin lymphoma determined by local hematopathology review * One of the following: * Age \>= 60 years * Age \< 60 years but unsuitable for standard chemotherapy because of a cardiac ejection fraction of \< 50%, a pulmonary diffusion capacity \< 80%, or a creatinine clearance \>= 30 and \< 60 mL/min, or refused standard chemotherapy despite efforts to convince them otherwise * Requirement for systemic chemotherapy: all stages except IA (not bulky disease), if involved field is considered radiotherapy (RT) curative * Previously untreated with either chemotherapy, radiation therapy or either brentuximab vedotin or nivolumab, or another check point inhibitor * Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0, 1, or 2 * Absolute neutrophil count (ANC) \>= 1500/mm\^3, unless secondary to bone marrow involvement; obtained =\< 7 days prior to registration * Leukocytes \>= 3,000/mm\^3, obtained =\< 7 days prior to registration * Platelet count \>= 100,000/mm\^3, obtained =\< 7 days prior to registration * Hemoglobin \> 9.0 g/dL - unless determined by treating physician to be disease related, obtained =\< 7 days prior to registration * Total bilirubin =\< 1.5 x upper limit of normal (ULN), obtained =\< 7 days prior to registration * Aspartate aminotransferase (aspartate transaminase \[AST\]) =\< 2.5 x ULN, obtained =\< 7 days prior to registration * Alanine aminotransferase (alanine transaminase \[ALT\]) =\< 2.5 x ULN, obtained =\< 7 days prior to registration * Creatinine =\< 2.0 mg/dL, obtained =\< 7 days prior to registration * Amylase and/or lipase =\< 1.5 x ULN, obtained =\< 7 days prior to registration * Women of childbearing potential must have a negative serum or urine pregnancy test (minimum sensitivity 25 IU/L or equivalent units of human chorionic gonadotropin \[HCG\]) within 24 hours prior to registration * Note: women of child-bearing potential (WOCBP) must use appropriate method(s) of contraception; WOCBP should use an adequate method to avoid pregnancy for 23 weeks (30 days plus the time required for nivolumab to undergo five half-lives) after the last dose of investigational drug; men who are sexually active with WOCBP must use any contraceptive method with a failure rate of less than 1% per year; men receiving nivolumab and who are sexually active with WOCBP will be instructed to adhere to contraception for a period of 31 weeks after the last dose of investigational product; women who are not of childbearing potential (i.e., who are postmenopausal or surgically sterile) as well as azoospermic men do not require contraception; should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform her treating physician immediately * Willing to return to enrolling institution for follow-up (during the active monitoring phase of the study) * Note: during the active monitoring phase of a study (i.e., active treatment and observation), participants must be willing to return to the consenting institution for follow-up * Ability to understand and willingness to sign an informed written consent * Provide blood and tissue samples for mandatory correlative research purposes Exclusion Criteria: * Any of the following because this study involves an investigational agent whose genotoxic, mutagenic and teratogenic effects on the developing fetus and newborn are unknown: * Pregnant women * Nursing women * Men or women of childbearing potential who are unwilling to employ adequate contraception * Co-morbid systemic illnesses or other severe concurrent disease which, in the judgment of the investigator, would make the patient inappropriate for entry into this study or interfere significantly with the proper assessment of safety and toxicity of the prescribed regimens * Active, known or suspected autoimmune disease; note: subjects are permitted to enroll if they have vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger * Use of systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications =\< 14 days of registration; Note: Inhaled or topical steroids and adrenal replacement doses \> 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease * Immunocompromised patients, patients with known history of testing positive for human immunodeficiency virus (HIV) or known acquired immunodeficiency syndrome (AIDS) and currently receiving antiretroviral therapy, patients with a prior history of known or suspected autoimmune disease, active hepatitis B virus surface antigen (HBV sAg+), active hepatitis C (if antibody \[Ab\]+ then polymerase chain reaction \[PCR\]+) indicating acute or chronic infection, and/or history of interstitial lung disease * Allergy to brentuximab vedotin and/or nivolumab * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Receiving any other investigational agent which would be considered as a treatment for the primary neoplasm * Have had prior chemotherapy or radiotherapy for Hodgkin lymphoma * Have received either of the study drugs * \< 60 years who are considered candidates for standard chemotherapy * \>= grade 2 peripheral neuropathy * Other active malignancy =\< 2 years prior to registration, unless treated with curative intent; EXCEPTIONS: non-melanotic skin cancer or carcinoma-in-situ of the cervix; NOTE: if there is a history or prior malignancy, they must not be receiving other specific treatment for their cancer * Active central nervous system (CNS) involvement or leptomeningeal metastases involvement * Known history of pancreatitis