A Trial Comparing Chemotherapy Versus Novel Immune Checkpoint Inhibitor (Pembrolizumab) Plus Chemotherapy in Treating Relapsed/Refractory Classical Hodgkin Lymphoma

Inclusion Criteria: * Patient must be \>= 5 years of age and =\< 75 years of age * Patient must have relapsed/refractory classical Hodgkin lymphoma (R/R cHL) after frontline (first line of chemotherapy) as evidenced by Fludeoxyglucose F-18 (FDG) avid on PET scan. If regimen is adjusted based on PET2 results or toxicity during frontline therapy, this will only be considered one line of therapy. Prior checkpoint inhibitor is completed \> 6 months prior to randomization * Patients \>= 18 years of age must have an Eastern Cooperative Oncology Group (ECOG) Performance Status of 0-2. Pediatric patients (16-17 years of age) must have a Karnofsky performance level \>= 50%. Pediatric patients (5-15 years of age) must have a Lansky performance level \>= 50 * Patient must be deemed fit for high dose chemo and autologous stem cell transplant * Patient must have the ability to understand and the willingness to sign a written informed consent document. Pediatric patients (\< 18 years of age) and patients with impaired decision-making capacity (IDMC) who have a legally authorized representative (LAR) or caregiver and/or family member available will also be considered eligible. Child assent must be obtained as appropriate in accordance with institutional guidelines * Absolute neutrophil count (ANC) \>= 1000/mcL (must be obtained =\< 7 days prior to randomization) * If disease includes marrow involvement or hypersplenism, please reference the below revised requirement: * ANC \>= 500/mcL * Platelets \>= 75,000/mcL (must be obtained =\< 7 days prior to randomization) * If disease includes marrow involvement or hypersplenism, please reference the below revised requirement: * Platelets \>= 25,000/mcL * Total bilirubin =\< 2x institutional upper limit of normal (ULN) (must be obtained =\< 7 days prior to randomization) * Aspartate aminotransferase (AST)(serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT)(serum glutamate pyruvate transaminase \[SGPT\]) =\< 3.0 x institutional ULN for age (must be obtained =\< 7 days prior to randomization) * Glomerular filtration rate (GFR) \>= 50 mL/min/1.73m\^2 73m\^2 for patients \>= 18 years of age (must be obtained =\< 7 days prior to randomization) * Pediatric patients (\< 18 years old) must have a creatinine clearance OR radioisotope GFR \>= 70 mL/min/1.73 m\^2 OR serum creatinine below the maximum based on age/gender as follows (derived from the Schwartz formula for estimating GFR utilizing child length and stature data published by the Centers for Disease Control and Prevention \[CDC\]): * Age: 5 to \< 6 years; Maximum Serum Creatinine (mg/dL): Male (0.8); Female (0.8) * Age: 6 to 10 years; Maximum Serum Creatinine (mg/dL): Male (1.0); Female (1.0) * Age: 10 to 13 years; Maximum Serum Creatinine (mg/dL): Male (1.2); Female (1.2) * Age: 13 to \< 16 years; Maximum Serum Creatinine (mg/dL): Male (1.5); Female (1.4) * Age: \>= 16 years; Maximum Serum Creatinine (mg/dL): Male (1.7); Female (1.4) * Patient must have a left ventricular ejection fraction (LVEF) \>= 50%, as measured by echocardiogram, multi-gated acquisition (MUGA) scan, or functional cardiac imaging scan (or a shortening fraction of \>= 27% for patients \< 18 years of age only) * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months of randomization are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * Patients \>= 18 years of age with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, patients should be class 2B or better Exclusion Criteria: * Patient must not be pregnant or breast-feeding due to the potential harm to an unborn fetus and possible risk for adverse events in nursing infants with the treatment regimens being used. * All patients of childbearing potential must have a blood test or urine study within 14 days prior to randomization to rule out pregnancy. * A patient of childbearing potential is defined as anyone, regardless of sexual orientation or whether they have undergone tubal ligation, who meets the following criteria: 1) has achieved menarche at some point, 2) has not undergone a hysterectomy or bilateral oophorectomy; or 3) has not been naturally postmenopausal (amenorrhea following cancer therapy does not rule out childbearing potential) for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months) * Patients of childbearing potential and/or sexually active patients must not expect to conceive or father children by using an accepted and effective method(s) of contraception or by abstaining from sexual intercourse for the duration of their participation in the study. Additionally, patients of childbearing potential must continue contraception measures for 4 months after the last dose of pembrolizumab, 6 months after the last dose of vinorelbine and for 3 months after the last dose of bendamustine. Male patients must continue contraception measures for 6 months after the last dose of ifosfamide and for 3 months after the last dose of vinorelbine. Patients must also not breastfeed while on study treatment and for 4 months after the last dose of Pembrolizumab and 9 days after the last dose of vinorelbine * Patient must not have a history of (non-infectious) pneumonitis/interstitial lung disease that required steroids or have current pneumonitis/interstitial lung disease * Patient must not have the following symptomatic autoimmune disorders: rheumatoid arthritis, systemic progressive sclerosis (scleroderma), systemic lupus erythematosus, Sjögren's syndrome, or autoimmune vasculitis (e.g., Wegener's Granulomatosis), or conditions of immunosuppression that require current ongoing treatment with systemic corticosteroids (or other systemic immunosuppressants), including oral steroids (i.e., prednisone, dexamethasone). Patients who discontinue use of these classes of medication for at least 2 weeks prior to randomization eligible if, in the judgment of the treating physician investigator, the patient is not likely to require resumption of treatment with these classes of drugs during the study. * Replacement doses of steroids for patients with adrenal insufficiency are allowed. * Additionally, patients must not have an autoimmune disease that is felt by the treating physician to have the potential to be exacerbated by checkpoint inhibition