Ipilimumab and Rituximab in Treating Patients With Relapsed or Refractory B-cell Lymphoma

Inclusion Criteria: * Previously treated, histologically confirmed cluster of differentiation (CD)20+ B cell lymphoma; bone marrow biopsies as the sole means of diagnosis are not acceptable, but they may be submitted in conjunction with nodal biopsies or extra nodal biopsies; fine needle aspirates are not acceptable * All patients must be informed of the investigative nature of the clinical trial and give written informed consent in accordance with institutional and federal guidelines * Able to adhere to the study visit schedule and other protocol requirements * Karnofsky \>= 70% * Life expectancy expected to be greater than 3 months * Leukocytes \>= 3,000/mcL * Absolute neutrophil count \>= 1,000/mcL * Platelets \>= 50,000/mcL * Total bilirubin =\< 2.0 x institutional upper limit of normal * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal * Serum creatinine =\< 2.0 x upper limit of normal OR calculated creatinine clearance \>= 30 ml/min/1.73 M\^2 by the modified Cockcroft and Gault formula OR creatinine clearance \>= 30 mL/min obtained from a 24-hour urine collection * At least one measurable lesion according to international workshop lymphoma response criteria; there must be measurable lymphadenopathy to follow with serial exam and/or imaging * All previous cancer therapy, including radiation, hormonal therapy and surgery, must have been discontinued at least 4 weeks prior to treatment in this study * Patients must have evidence of progression of disease during or after last treatment * Submission of original biopsy for review and verification by participating center hematopathologist * Disease free of prior malignancies for \>= 3 years with exception of currently treated basal cell, squamous cell carcinoma of the skin, or carcinoma "in situ" of the cervix or breast Exclusion Criteria: * Patients who have had chemotherapy or radiotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier * Patients with a history of prior treatment with ipilimumab * Patients with a history of prior treatment with an anti-programmed cell death (PD) 1 antibody, CD137 agonist or other immune activating therapy such as anti-CD 40 antibody are excluded unless 5 half-lives of the agent (minimum of 8 weeks) have intervened since the therapy; patients who have received prior vaccine therapy are eligible * Patients who are receiving any other investigational agents * Autoimmune disease: patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn's disease, are excluded from this study, as are patients with a history of symptomatic disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[e.g., Wegener's granulomatosis\]); central nervous system (CNS) or motor neuropathy considered of autoimmune origin (e.g. Guillain-Barre syndrome and myasthenia gravis, multiple sclerosis) * Patients with known immune impairment who may be unable to respond to anti-cytotoxic T-lymphocyte antigen 4 (CTLA 4) antibody * Patients with known uncontrolled brain metastases are excluded; however, patients with stable brain disease (off corticosteroids) at least 2 weeks after completion of appropriate therapy for their brain metastases are eligible * History of allergic reactions attributed to compounds of similar chemical or biologic composition to rituximab * Patients on systemic corticosteroids (except for patients on stable doses of hormone replacement therapy such as hydrocortisone), or other immunosuppressants (e.g., infliximab, mycophenolate mofetil) are excluded * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Patients with chronic human immunodeficiency virus (HIV), hepatitis B or hepatitis C infections are excluded * Pregnant women are excluded from this study * HIV-positive patients on combination antiretroviral therapy are ineligible * Rituximab within six weeks