Pembrolizumab in Relapsed and Refractory Gray-Zone Lymphoma (GZL), Primary Central Nervous System Lymphoma (PCNSL), and Other Extranodal Diffuse Large B-cell Lymphomas

* INCLUSION CRITERIA: * Patients must have a diagnosis of B-cell lymphoma confirmed by Laboratory of Pathology, National Cancer Institute (NCI), that is relapsed from or refractory to prior therapy as follows: * Cohort 1: B-cell lymphoma, unclassifiable, with features intermediate between diffuse large B-cell lymphoma and classical Hodgkin lymphoma (i.e., Gray-zone lymphoma or GZL) * Cohort 2: Extranodal diffuse large B-cell lymphoma involving one or more of the specified extranodal sites (i.e., extranodal diffuse large B-cell lymphoma (DLBCL). The following subtypes are included (they do not have to be confirmed as non-germinal center (non-GCB) subtype for study entry): * Primary central nervous system (CNS) lymphoma (PCNSL) * Primary testicular lymphoma (PTL) * Primary breast lymphoma (PBL) * Primary cutaneous DLBCL, leg-type * Intravascular large B-cell lymphoma (IVBCL) * Diffuse large B-cell, not otherwise specified (NOS), activated B-cell type, involving 1 or more extranodal site NOTE: For GZL, diagnosis will be in accordance with the 2016 World Health Organization classification of lymphoid malignancies. Patients diagnosed with other extranodal DLBCL subtypes or that are not otherwise specified (NOS) must involve at least 1 extranodal site and must be considered non-GCB by local immunohistochemistry algorithms. Cases that are non-GCB by the Hans criteria are considered eligible as well as cases of DLBCL that are both cluster of differentiation 10 positive (CD10+) and multiple myeloma 1 positive (MUM1+). * Evaluable disease by clinical exam (i.e., palpable lymphadenopathy, measurable skin lesions, etc.), laboratory assessment (i.e., lymphoma involvement of bone marrow or peripheral blood by morphology, cytology or flow cytometry), and/or imaging (measurable lymph nodes or masses on computed tomography (CT) or magnetic resonance imaging (MRI) and/or evaluable fluorine-18-deoxyglucose (FDG)-avid lesions on positron emission tomography (PET) * Adequate tumor tissue (archival or fresh) must be available for correlative studies. NOTE: Tumor tissue may be from any previously collected tissue and adequacy is at the discretion of the Principal Investigator. If prior tissue is not available, patient must be willing to undergo baseline tumor biopsy. * Be 18 years of age or older on day of signing informed consent * Adequate performance status (PS) as follows: * Patients greater than or equal to 18 years must have Eastern Cooperative Oncology Group (ECOG) 0-1 (and Karnofsky greater than or equal to 60%) NOTE: Patients greater than or equal to 18 years with an ECOG PS of 2 and Karnofsky greater than or equal to 60 will be considered eligible at the discretion of the Principal Investigator if decreased ECOG performance status is felt to be related to residual neurologic deficits caused by CNS disease involvement that are not progressive or anticipated to cause clinical management problems during study participation. \- Adequate organ function as evidenced by the following laboratory parameters (unless related to lymphoma infiltration at the discretion of the investigator): * Absolute neutrophil count (ANC) greater than or equal to 750 /mcL * Platelets greater than or equal to 50,000/mcL (transfusions not permitted) * Hemoglobin greater than or equal to 9 g/dL (transfusions permitted) * Serum creatinine: Adults: less than or equal to 1.5 times upper limit of normal (ULN). Children: age greater than or equal to 14: less than or equal to 1.5 mg/dL OR measured or calculated creatinine clearance (glomerular filtration rate (GFR) can also be used in place of creatinine or creatinine clearance (CrCl): Greater than or equal to 30 mL/min/1.73 m(2) for subject with creatinine levels \> 1.5 times institutional ULN (CrCl should be calculated per institutional standard) --Serum total bilirubin less than or equal to 1.5 times ULN OR Direct bilirubin less than or equal to ULN for patients with total bilirubin levels \> 1.5 ULN * Aspartate aminotransferase (AST) serum glutamic oxaloacetic transaminase (SGOT) and alanine aminotransferase (ALT) serum glutamic pyruvic transaminase (SGPT) less than or equal to 3 times ULN (less than or equal to 5 X ULN if liver involvement) \- The effects of pembrolizumab on the developing human fetus are unknown. For this reason, the following measures apply: * Women of childbearing potential must have a negative serum or urine pregnancy test within 72 hours prior to the first dose of pembrolizumab. * Men and women of childbearing potential (WOCBP) who are sexually active must agree to adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and for at least 120 days after the last dose of pembrolizumab. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. * Participants must not be planning to conceive or father children within the projected duration of the trial, starting with the pre-screening/screening visit through 120 days after the last dose of pembrolizumab. * WOCBP is defined as any female who has experienced menarche and who has not undergone successful surgical sterilization or who is not postmenopausal. * Ability of patient or Legally Authorized Representative (LAR) to understand and the willingness to sign a written informed consent document EXCLUSION CRITERIA: * Patients with DLBCL who best fit the criteria of Epstein-Barr virus (EBV)+ DLBCL, NOS are not eligible * Current or prior anti-cancer treatment prior to the first dose of pembrolizumab as defined below: * Chemotherapy, targeted small molecule therapy, or other anti-cancer treatment not otherwise specified below within 2 weeks * Radiation therapy within 2 weeks * Anti-cancer monoclonal antibody (mAb) treatment within 4 weeks * Use of an investigational agent (e.g., biologic, drug, or other) within 4 weeks * Allogeneic stem cell transplant within 100 days * Prior therapy with an anti-programmed cell death protein 1 (PD-1), anti-programmed death ligand 1 (PD-L1), or anti-programmed death ligand 2 (PD-L2) agent at any time * No current use of systemic corticosteroids at physiologic doses \> 10 mg/day of dexamethasone or equivalent are permitted. Patients receiving current systemic steroids must be on a stable steroid dose (i.e., less than or equal to 10 mg/day of dexamethasone or equivalent at the same dose for at least 7 days). Patients who recently discontinued systemic steroids must have completed them at least 7 days prior to entry. * Uncontrolled intercurrent illness including, but not limited to the following that may limit interpretation of results or that could increase risk to the patient at the discretion of the investigator: * Active autoimmune disease that has required systemic treatment in the past 2 years (i.e., with use of disease modifying agents, corticosteroids or immunosuppressive drugs). NOTE: Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment. * History of (non-infectious) pneumonitis that required steroids, evidence of interstitial lung disease or active, non-infectious pneumonitis. * Clinically significant history of liver disease, including viral or other hepatitis, current alcohol abuse, or cirrhosis; as well as active infection with hepatitis B virus (HBV) or hepatitis C virus (HCV): ---Patients with occult or prior HBV infection (defined as positive total hepatitis B core antibody \[HBcAb\] and negative HBsAg) may be included if HBV deoxyribonucleic acid (DNA) is undetectable. * Uncontrolled and/or symptomatic thyroid disease * Active graft-vs-host disease (GVHD) requiring treatment or any history of greater than or equal to grade II acute GVHD * Seizure activity within the past 4 weeks * Known mental or physical illness that would interfere with cooperation with the requirements of the trial or confound the results or interpretation of the results of the trial and, in the opinion of the treating investigator, would make the patient inappropriate for entry into the study. * Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with pembrolizumab, breastfeeding must be discontinued if the mother is treated with pembrolizumab * Received a live vaccine within 30 days of planned start of study therapy. NOTE: Seasonal influenza vaccines for injection are generally inactivated flu vaccines and are allowed; however intranasal influenza vaccines (e.g., Flu-Mist) are live attenuated vaccines, and are not allowed. * History of allergic reactions attributed to compounds of similar chemical or biologic composition to pembrolizumab unless felt to be in the best interests of the patient in the opinion of the investigator * Known additional malignancy that requires active systemic treatment