Brentuximab Vedotin With or Without Nivolumab in Treating Patients With Relapsed or Refractory CD30+ Lymphoma

Inclusion Criteria: * Relapsed or refractory CD30+ lymphoma that has either achieved \< PR to brentuximab vedotin (minimum of 2 cycles), progressed while receiving brentuximab vedotin, or progressed within 6 months of the last dose of brentuximab vedotin * Documented expression of CD30 on tumor cells * Absolute neutrophil count (ANC) \> 1,000/uL * Platelets \> 50,000/uL * Serum creatinine \< 1.5 mg/dL OR creatinine clearance \> 60 mL/min * Bilirubin \< 1.5 x upper limit of normal (ULN) * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) \< 2.5 x ULN * Measurable disease by computed tomography (CT) or similar (e.g. magnetic resonance imaging \[MRI\]) criteria (\> 1.5 cm). (Patients with cutaneous lymphoma only require measurable disease by Olsen Criteria) * Age \>= 18 yrs at the time of the first dose of study drug * Resolution of all non-hematologic brentuximab vedotin-related and nivolumab-related adverse events (AEs) to \< Grade 2 * All patients must be informed of the investigational nature of this study and have given written consent in accordance with institutional and federal guidelines * Patients must be anticipated to complete at least 2 cycles of chemotherapy on study * Expected survival if untreated of \> 90 days Exclusion Criteria: * Prior transplant within 100 days * Radioimmunotherapy within 12 weeks * Known human immunodeficiency virus (HIV) or hepatitis B positivity or prior progressive multifocal leukoencephalopathy (PML) * Active infection or other medical condition which would preclude treatment in the opinion of the principal investigator; this would include a corrected diffusing capacity of the lungs for carbon monoxide (DLCO) of \< 60% predicted or symptomatic interstitial lung disease * Eastern Cooperative Oncology Group (ECOG) performance status \> 2 * Known active central nervous system (CNS) involvement * Peripheral neuropathy \> grade 1 if due to brentuximab vedotin or any peripheral neuropathy \> grade 2 * Intolerance to brentuximab vedotin * Concurrent use of other anti-cancer agents or experimental treatments * No current or prior autoimmune disease with the exception of vitiligo and autoimmune alopecia (Arm B only) * Pregnancy or breastfeeding; (females of childbearing potential must have a negative serum or urine beta human chorionic gonadotropin \[beta-hCG\] pregnancy test result within 7 days prior to the first dose of brentuximab vedotin; females with false positive results and documented verification that the patient is not pregnant are eligible for participation; females of non-childbearing potential are those who are postmenopausal greater than 1 year or who have had a bilateral tubal ligation or hysterectomy; females of childbearing potential and males who have partners of childbearing potential must agree to use 2 effective contraceptive methods during the study and for 6 months following the last dose of brentuximab vedotin or 6 months following the last dose of nivolumab, whichever is later)