Nivolumab and Ipilimumab in Treating Patients With HIV Associated Relapsed or Refractory Classical Hodgkin Lymphoma or Solid Tumors That Are Metastatic or Cannot Be Removed by Surgery

Inclusion Criteria: * Participants must have histologically confirmed solid tumor malignancy that is metastatic or unresectable and for which standard curative or palliative measures do not exist or are no longer effective; participants with uncontrolled Kaposi sarcoma are permitted (KS must be increasing despite HAART and HIV suppression for greater than or equal to 2 months, or stable KS despite HAART for greater than or equal to 3 months) * For participants in the 24 participant solid tumor cohort, only those histologies not known to respond to single agent nivolumab (such as pancreas, prostate, and microsatellite stable \[MSS\] colorectal cancer) will be excluded * For participants in the relapsed refractory HIV-cHL expansion cohort, participants must have histologically confirmed, relapsed/refractory (defined as relapsed/refractory to one or greater lines of therapy) HIV-associated classical Hodgkin lymphoma * HIV-1 infection, as documented by any federally approved, licensed HIV rapid test performed in conjunction with screening (or enzyme linked immunosorbent assay \[ELISA\], test kit, and confirmed by Western blot or other approved test); alternatively, this documentation may include a record demonstrating that another physician has documented the participant's HIV status based on either: 1) approved diagnostic tests, or 2) the referring physician's written record that HIV infection was documented, with supporting information on the participant's relevant medical history and/or current management of HIV infection * Participants must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for non-nodal lesions and short axis for nodal lesions) as \>= 20 mm with conventional techniques or as \>= 10 mm with spiral computed tomography (CT) scan, magnetic resonance imaging (MRI), or calipers by clinical exam; scans must have been performed within 4 weeks prior to registration; Note: for participants with Kaposi sarcoma, the following apply: at least five measurable cutaneous KS lesions or any number of lesions with systemic unresectable disease with no previous local radiation, surgical, or intralesional cytotoxic therapy that would prevent response assessment * Prior therapy for metastatic disease permitted; at least 4 weeks must have elapsed since prior chemotherapy or biological therapy, 6 weeks if the regimen included carmustine (BCNU) or mitomycin C; radiotherapy must be completed at least 4 weeks prior to registration * Age \> 18 years, because no dosing or AE data are currently available on the use of ipilimumab in combination with nivolumab in participants \<18 years of age, children are excluded from this study. * Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky \>= 70%) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Leukocytes \>= 2,000/mm\^3 (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Absolute neutrophil count \>= 1,000/mm\^3 (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Platelets \>= 75,000/mm\^3 (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) =\< 3 x ULN for subjects with Gilbert's disease or with atazanavir- or indinavir-induced unconjugated hyperbilirubinemia without aspartate aminotransferase (AST) or alanine aminotransferase (ALT) elevation and must have a total bilirubin less than 3.0 mg/dL (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Serum lipase and amylase \< 1.5 x ULN (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: AST (serum glutamic oxaloacetic transaminase \[SGOT\])/ALT (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x ULN (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Creatinine \< 1.5 UNL or creatinine clearance (CrCl) \> 50 ml/min (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Hemoglobin \>= 9 g/dL (within 2 weeks prior to enrollment) * PARTICIPANTS NOT ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Serum albumin \>= 2.8 g/dL (within 2 weeks prior to enrollment) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Leukocyte count: no lower limit (within 2 weeks prior to enrollment) (participants may receive granulocyte colony stimulating factor \[GCSF\] and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Absolute neutrophil count: \>= 1,000/mm\^3, unless decreased due to bone marrow involvement with lymphoma (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Platelets: \>= 75,000/mm\^3, unless decreased due to bone marrow involvement with lymphoma (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Hemoglobin: \>= 9 g/dL unless bone marrow involvement secondary to Hodgkin lymphoma is present (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Total bilirubin: =\< 1.5 x institutional upper limit of normal (ULN), or =\< 3 x ULN for participants with Gilbert's disease or with atazanavir- or indinavir-induced unconjugated hyperbilirubinemia without AST or ALT elevation, and must have a total bilirubin less than 3.0 mg/dL) (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Serum lipase and amylase \< 1.5 x ULN (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: AST (SGOT)/ALT (SGPT): =\< 3 x ULN (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * PARTICIPANTS ON THE HODGKIN LYMPHOMA EXPANSION COHORT: Creatinine: \< 1.5 x upper normal limit (UNL) or CrCl \> 50ml/min (within 2 weeks prior to enrollment) (participants may receive GCSF and transfusions to meet these parameters) * HIV viral load should be well suppressed, defined as below the limit of detection of the local assay or below 75 copies/mL by Food and Drug Administration (FDA)-approved assays, within 4 weeks prior to registration * CD4 counts: * For Stratum 1: CD4+ cell count greater than 200 cells/mm\^3 obtained within 2 weeks prior to enrollment at any United States (U.S.) laboratory that has a clinical laboratory improvement amendments (CLIA) certification or its equivalent * For Stratum 2: CD4 cell count between 100-200 cells/mm\^3 obtained within 2 weeks prior to enrollment at any U.S. laboratory that has a clinical laboratory improvement amendments (CLIA) certification or its equivalent * Expansion Cohort: CD4 cell count for this cohort will be specified once Stratum 1 and Stratum 2 have completed enrollment * Solid Tumor Expansion Cohort: CD4+ cell count greater than 200 cells/mm\^3 obtained within 2 weeks prior to enrollment at any U.S. laboratory that has a clinical laboratory improvement amendments (CLIA) certification or its equivalent * cHL Cohort: CD4 cell count of at least 100 cells/mm\^3 * Participants must be purified protein derivative (PPD) negative; alternatively, the QuantiFERON-tuberculosis (TB) Gold In-Tube (QFT-GIT) assay can be used; an individual is considered positive for M. tuberculosis infection if the IFN-gamma response to TB antigens is above the test cut-off (after subtracting the background IFN-gamma response in the negative control); the result must be obtained within 12 weeks prior to enrollment; PPD positive (or Quantiferon assay positive) participants are permitted if prophylaxis has been completed prior to enrollment * The effects of nivolumab and ipilimumab on the developing human fetus are unknown; for this reason and because other therapeutic agents used in this trial are known to be teratogenic, women of childbearing potential (WOCBP) and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation; WOCBP should use an adequate method to avoid pregnancy for 6 months after the last dose of investigational drug; women of childbearing potential must have a negative serum or urine pregnancy test (minimum sensitivity 25 IU/L or equivalent units of human chorionic gonadotropic \[HCG\]) within 72 hours prior enrollment and the start of nivolumab; women must not be breastfeeding; men who are sexually active with WOCBP must use any contraceptive method with a failure rate of less than 1% per year; men receiving nivolumab and who are sexually active with WOCBP will be instructed to adhere to contraception for a period of 31 weeks after the last dose of investigational product; women who are not of childbearing potential (i.e., who are postmenopausal or surgically sterile as well as azoospermic men) do not require contraception; WOCBP receiving nivolumab will be instructed to adhere to contraception for a period of 6 months after the last dose of investigational product; men receiving nivolumab and who are sexually active with WOCBP will be instructed to adhere to contraception for a period of 31 weeks after the last dose of investigational product; should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she (or the participating partner) should inform the treating physician immediately * Participants MUST receive appropriate care and treatment for HIV infection, including antiretroviral medications when clinically indicated, and should be under the care of a physician experienced in HIV management; participants will be eligible regardless of antiretroviral medication (including no antiretroviral medication) provided there is no intention to initiate therapy or the regimen has been stable for at least 4 weeks with no intention to change the regimen within 12 weeks following enrollment * Participants who have hepatitis C (both reactive anti-hepatitis C virus \[HCV\] antibody and detectable HCV ribonucleic acid \[RNA\]) and hepatitis B (hepatitis B surface antigen \[HBsAg\] positive and anti-hepatitis B core \[HBc\]-total positive), may be enrolled, provided total bilirubin is =\< 1.5 x institutional ULN, and AST (SGOT) and ALT (SGPT) must be =\< 3 X institutional upper limit of normal, and hepatitis B virus (HBV) deoxyribonucleic acid (DNA) \< 100 IU/mL (if hepatitis B positive) within 2 weeks prior to enrollment * Ability to understand and to sign a written informed consent document * Criteria for Solid Tumor Expansion and Lymphoma Cohorts: * Inclusion and exclusion criteria for this cohort are the same as above, with the following rule for CD4 count based on tolerability in Phase I; if, participants with lymphocyte T CD4 count between 100-200/mm\^3 (Stratum 2) are shown to tolerate treatment in the Phase I dose de-escalation portion at the same dose level as those with CD4 counts \> 200/mm\^3 (Stratum 1), participants in the expansion cohort with CD4 counts \>= 100/mm\^3 are permitted; otherwise, the expansion is open to all solid tumor patients except those whose tumors are known not to respond to nivolumab (pancreas, prostate and MSS colon cancer); for the relapsed refractory HIV-cHL cohort, participants with CD4 count \>= 100/mm\^3 are permitted Exclusion Criteria: * Participants who have received any other investigational agents within the 4 weeks prior to enrollment; concurrent radiation therapy is not permitted, except palliative (limited-field) radiation therapy, if all of the following criteria are met: * Repeat imaging demonstrates no new sites of bone metastases * The lesion being considered for palliative radiation is not a target lesion * Participants with known brain metastases or leptomeningeal metastases must be excluded unless they qualify for enrollment as described below because of poor prognosis and concerns regarding progressive neurologic dysfunction that would confound the evaluation of neurologic and other adverse events; participants with brain metastases are permitted if metastases have been treated and there is no magnetic resonance imaging (MRI) evidence of progression for at least 4 weeks or more after treatment is complete and within 4 weeks prior to the first dose of nivolumab administration * History of allergic reactions attributed to compounds of similar chemical or biologic composition to ipilimumab, nivolumab, or other agents used in study, or history of severe hypersensitivity reaction to any monoclonal antibody * Participants should be excluded if they have a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 2 weeks of study drug administration; these drugs may interfere with the activity of ipilimumab and nivolumab if administered at the time of the first ipilimumab dose; inhaled or topical steroids and adrenal replacement doses =\< 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease; participants are permitted to use topical, ocular, intra-articular, intranasal, and inhalational corticosteroids (with minimal systemic absorption); physiologic replacement doses of systemic corticosteroids are permitted, even if \>= 10 mg/day prednisone equivalents; a brief course of corticosteroids for prophylaxis (e.g., contrast dye allergy) or for treatment of non-autoimmune conditions (e.g., delayed-type hypersensitivity reaction caused by contact allergen) is permitted; use of anabolic steroids is permitted * Participants with clinical or radiographic evidence of pancreatitis are excluded * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Participants should be excluded if they have had prior treatment with an anti-PD-1, anti-programmed cell death ligand 1 (PD-L1), anti-programmed cell death ligand 2 (PD-L2), anti-CTLA-4 antibody, or any other antibody or drug specifically targeting T-cell co-stimulation or immune checkpoint pathways; prior immune modulating therapy including vaccines may be eligible; prior immune events must be evaluated and the risk for new events which may represent continued sub clinical disease or a new process at previously damaged site or immune potentiation (e.g. ipilimumab followed by IL2 causing bowel perforation, ipilimumab followed by indoleamine 2,3-dioxygenase \[IDO\] inhibitor resulting in clinical hypophysitis); please keep in mind that inflammatory events may occur weeks to months following the last dose of ipilimumab and possibly nivolumab; assessment of potential effects of prior therapy should include: * Immune status * Organ damage * Risk of autoimmunity * Immunopotentiation * The participant has not recovered to baseline or Common Terminology Criteria for Adverse Events (CTCAE) =\< grade 1 from toxicity due to all prior therapies except =\< grade 2 alopecia, neuropathy, and other non-clinically significant adverse events (AEs) * The participant has a primary brain tumor * Participant has \>= grade 2 diarrhea (participants with grade 1 diarrhea are eligible provided stool for ova/parasites and stool cryptosporidium studies are negative) * Opportunistic infection within the last 3 months * Participants with active autoimmune disease or history of autoimmune disease that might recur, which may affect vital organ function or require immune suppressive treatment including systemic corticosteroids, should be excluded; these include but are not limited to patients with a history of immune related neurologic disease, multiple sclerosis, autoimmune (demyelinating) neuropathy, Guillain-Barre syndrome, myasthenia gravis; systemic autoimmune disease such as systemic lupus erythematosus (SLE), connective tissue diseases, scleroderma, inflammatory bowel disease (IBD), Crohn's, ulcerative colitis, hepatitis; and participants with a history of toxic epidermal necrolysis (TEN), Stevens-Johnson syndrome, or phospholipid syndrome should be excluded because of the risk of recurrence or exacerbation of disease; participants with vitiligo, endocrine deficiencies including thyroiditis managed with replacement hormones including physiologic corticosteroids are eligible; participants with rheumatoid arthritis and other arthropathies, Sjogren's syndrome and psoriasis controlled with topical medication and participants with positive serology, such as antinuclear antibodies (ANA), anti-thyroid antibodies should be evaluated for the presence of target organ involvement and potential need for systemic treatment but should otherwise be eligible; participants are permitted to enroll if they have vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger (precipitating event) * Participants who have had evidence of Clostridium (C.) difficile infection, active or acute diverticulitis, intra-abdominal abscess, intra-abdominal abscess, gastrointestinal (GI) obstruction and abdominal carcinomatosis, which are known risk factors for bowel perforation, should be evaluated for the potential need for additional treatment before coming on study * cHL COHORT ONLY: history of allogeneic transplant