Pomalidomide and Dexamethasone in Treating Patients With Relapsed or Refractory Primary Central Nervous System Lymphoma or Newly Diagnosed or Relapsed or Refractory Intraocular Lymphoma

Inclusion Criteria: * Relapsed or refractory primary central nervous system (CNS) diffuse large B cell lymphoma (PCNSDLBCL) with a CNS lesion, with cerebrospinal fluid (CSF) relapse with positive CSF cytology, or with ocular relapse with positive ocular tissue biopsy; NOTE: tissue biopsy is not absolutely necessary for CNS tumor unless clinical and radiologic findings strongly suggest other etiologies as per treating physician; initial diagnosis must be made by tissue biopsy; NOTE: patients with B-cell lymphoma with features intermediate between diffuse large B-cell lymphoma and Burkitt lymphoma are also eligible for the protocol as long as they meet other criteria; patients with typical Burkitt lymphoma are not eligible * Relapsed/refractory primary vitreoretinal diffuse large B cell lymphoma (DLBCL) with a CNS lesion, with CSF relapse with positive CSF cytology, or with ocular relapse with positive ocular tissue biopsy; NOTE: tissue biopsy requirement of the CNS lesion is as outlined in bullet above * Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0, 1, 2 or 3 * Absolute neutrophil count (ANC) \>= 1000/uL * Platelets (PLT) \>= 100,000/uL * Total bilirubin =\< 1.5 x upper limit of normal (ULN) or if total bilirubin is \> 1.5 x ULN the direct bilirubin must be =\< 1.5 x ULN (=\< 0.45 mg/dL) * Aspartate aminotransferase (AST) =\< 3 x ULN * Creatinine =\< 2.5 x ULN * Females of reproductive potential must be willing to adhere to the scheduled pregnancy testing as required in the POMALYST REMS (TM) program * Able to take aspirin (81 or 325 mg) daily as prophylactic anticoagulation (patients intolerant to acetylsalicylic acid \[ASA\] may use warfarin or heparin) * Provide informed written consent * Willing to return to participating medical institutions for follow-up * Willing to provide tissue samples for correlative research purposes * Willing to be registered into the mandatory POMALYST REMS (TM) program, and willing and able to comply with the requirements of the POMALYST REMS (TM) program Exclusion Criteria: * Any of the following * Pregnant women * Nursing women * Men or women of childbearing potential who are unwilling to employ adequate contraception * The development of erythema nodosum if characterized by a desquamating rash while taking thalidomide or similar drugs * Uncontrolled infection * Therapy with myelosuppressive chemotherapy or biologic therapy \< 21 days prior to registration; NOTE: patients who have recovered from cytopenia related to previous treatment and meet criteria of this protocol will be eligible * Persistent toxicities \>= grade 3 from prior chemotherapy or biological therapy regardless of interval since last treatment * History of thromboembolic episodes =\< 3 months prior to registration * Other concurrent chemotherapy, immunotherapy, radiotherapy, or any ancillary therapy considered investigational (utilized for a non-Food and Drug Administration \[FDA\]-approved indication and in the context of a research investigation) * Immunodeficiency states including human immunodeficiency virus (HIV) infection * Active hepatitis B or C with uncontrolled disease; NOTE: a detailed assessment of hepatitis B/C medical history and risk factors must be done at screening for all patients; hepatitis B core immunoglobulin M antibody (HBcIgM Ab), hepatitis B surface antigen (HBsAg) and hepatitis C antibody screen (HCV Ab Scrn) w/reflex testing are required at screening for all patients with a positive medical history based on risk factors and/or confirmation of prior hepatitis B (HBV) infection * Active other malignancy requiring treatment that would interfere with the assessments of response of the lymphoma to protocol treatment * Inability to swallow or impairment of gastrointestinal function or gastrointestinal disease that may significantly alter the absorption of the drugs (e.g., ulcerative disease, uncontrolled nausea, vomiting, diarrhea, malabsorption syndrome or small bowel resection) that would preclude use of oral medications * Any severe and/or uncontrolled medical conditions or other conditions that, in the treating physician's opinion, could adversely impact their ability to participate in the study * Major surgery =\< 4 weeks prior to registration or have not recovered from side effects of such therapy * New York Heart Association classification III or IV