Pembrolizumab in Treating Patients With Relapsed or Refractory Stage IB-IVB Mycosis Fungoides or Sezary Syndrome

Inclusion Criteria: * This trial will include subjects with stage IB-IVB MF/SS (maximal stage since diagnosis will determine eligibility), and who have relapsed, are refractory, or progressed after at least one standard systemic therapy; current disease stage at time of entry will also be documented but will not be used for eligibility * Subjects must have the following minimum wash-out and adverse event (AE) recovery period from previous treatments without treatment between documentation of relapse/progression and enrollment of specifically: * \>= 2 weeks for local radiation therapy * \>= 4 weeks for systemic cytotoxic anticancer agents, anticancer investigational agents that are not defined as immunotherapy, or for tumor-targeting monoclonal antibodies (mAbs) with the exception of alemtuzumab for which the washout is at least 8 weeks * \>= 15 weeks for anti-cluster of differentiation (CD)137 or anti-cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) (including ipilimumab or any other antibody or drug specifically targeting T-cell co-stimulation or checkpoint pathways) * \>= 2 weeks from resolution (i.e., =\< grade 1 or at baseline) from AEs due to procedures performed or therapeutic agents administered * \>= 2 weeks for retinoids, interferons, vorinostat, romidepsin, denileukin diftitox, and therapeutic doses of oral corticosteroids (physiologic replacement doses of oral corticosteroids are allowed, topical corticosteroids are allowed) * \>= 2 weeks for phototherapy * \>= 1 week for topical therapy (including retinoid, nitrogen mustard, or imiquimod) * Have a performance status of 0 or 1 on the Eastern Cooperative Oncology Group (ECOG) performance scale * Life expectancy of at least 6 months * Performed within 10 days of treatment initiation: Leukocytes \>= 2,000/mcL * Performed within 10 days of treatment initiation: Absolute neutrophil count \>= 1,500/mcL * Performed within 10 days of treatment initiation: Platelets \>= 100,000/mcL * Performed within 10 days of treatment initiation: Hemoglobin \>= 9 g/dL OR \>= 5.6 mmol/L * Performed within 10 days of treatment initiation: Serum total bilirubin =\< 1.5 x upper limit of normal (ULN) OR direct bilirubin =\< ULN for subjects with total bilirubin levels \> 1.5 ULN * Performed within 10 days of treatment initiation: Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x ULN OR =\< 5 x ULN for patients with liver metastases * Performed within 10 days of treatment initiation: Serum creatinine =\< 1.5 x ULN OR measured or calculated creatinine clearance \>= 60 mL/min for subject with creatinine clearance (CrCl) levels \> 1.5 x institutional ULN (glomerular filtration rate \[GFR\] can also be used in place of creatinine or CrCl) * Performed within 10 days of treatment initiation: International normalized ratio (INR) or prothrombin time (PT) =\< 1.5 x ULN unless subject is receiving anticoagulant therapy in which case the INR or PT must be within the therapeutic range of intended use for the anticoagulant * Performed within 10 days of treatment initiation: Activated partial thromboplastin time (aPTT) =\< 1.5 x ULN unless subject is receiving anticoagulant therapy in which case the aPTT must be within the therapeutic range of intended use for the anticoagulant * Performed within 10 days of treatment initiation: Thyroid stimulating hormone (TSH) within institutional limits (ie: normal); if TSH is greater or less than institutional limits patients may participate if their thyroxine (T4) is within normal limits (WNL); patients may be on a stable dose of replacement thyroid medication; dose adjustments are allowed if needed * Patients must provide tissue from a punch biopsy of the skin at baseline, at the time of a clinical event (at the time of response, progression or appearance of a new lesion) and at the end of treatment; additional punch biopsies every 3 cycles are optional; an archival tissue sample is optional * Have measurable disease based on modified severity-weighted assessment tool (mSWAT); tumor lesions situated in a previously irradiated area are considered measurable if progression has been demonstrated in such lesions * Women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) from the time of the pre-study visit, through the course of the study and for 120 days after the last dose of study medication * Female patients of childbearing potential should have a negative urine or serum pregnancy within 72 hours prior to receiving the first dose of study medication; if the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required * Female patients of childbearing potential should be willing to use 2 methods of birth control or be surgically sterile, or abstain from heterosexual activity from the time of the pre-study visit, through the course of the study and for 120 days after the last dose of study medication; patients of childbearing potential are those who have not been surgically sterilized or have not been free from menses for \> 1 year * Should a woman become pregnant or suspect she is pregnant while she is participating in this study, she should inform her treating physician immediately; men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 4 months after completion of MK-3475 administration * Ability to understand and the willingness to sign a written informed consent document Exclusion Criteria: * Patients who have had chemotherapy or targeted small molecule therapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier * Note: patients with =\< grade 2 neuropathy are an exception to this criterion and may qualify for the study * Note: if patients received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting therapy * Patients who are currently participating in or have participated in a study of an investigational agent or using an investigational device within 4 weeks of the first dose of treatment * Has a diagnosis of immunodeficiency or is receiving therapeutic systemic steroid therapy or any other form of immunosuppressive therapy within 7 days prior to the first dose of trial treatment * Has had a prior monoclonal antibody within 4 weeks prior to study day 1 or who has not recovered (i.e., =\< grade 1 or at baseline) from AEs due to agents administered more than 4 weeks earlier * Has a known additional malignancy that is progressing or requires active treatment; exceptions include basal cell carcinoma of the skin, squamous cell carcinoma of the skin, or in situ cervical cancer that has undergone potentially curative therapy * Patients with known brain metastases should be excluded from this clinical trial * Patients with carcinomatous meningitis should also be excluded * Patients with previously treated brain metastases may participate provided they are stable (without evidence of progression by imaging for at least 4 weeks prior to the first dose of trial treatment and any neurologic symptoms have returned to baseline), have no evidence of new or enlarging brain metastases, and are not using steroids for at least 7 days prior to trial treatment * History of allergic reactions attributed to compounds of similar chemical or biologic composition to MK-3475 * Has an active autoimmune disease requiring systemic treatment within the past 3 months or a documented history of clinically severe autoimmune disease, or a syndrome that requires systemic steroids or immunosuppressive agents; patients with vitiligo or resolved childhood asthma/atopy would be an exception to this rule; patients that require intermittent use of bronchodilators or local steroid injections would not be excluded from the study; the use of physiologic doses of corticosteroids may be approved after consultation with the protocol principal investigator (PI) and Cancer Immunotherapy Trials Network (CITN); patients with hypothyroidism stable on hormone replacement or Sjogren's syndrome will not be excluded from the study * Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the patient's participation for the full duration of the trial, or is not in the best interest of the patient to participate, in the opinion of the treating investigator * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial * Has received prior therapy with an anti-programmed cell death (PD)-1, anti-PD-ligand (L)1, anti-PD-L2 * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, interstitial lung disease or active, non-infectious pneumonitis, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * History of other pulmonary disease such as emphysema or chronic obstructive pulmonary disease, (forced expiratory volume in one second \[FEV1\] \< 60% of predicted for height and age); pulmonary function tests (PFTs) are required in patients with prolonged smoking history or symptoms of respiratory dysfunction * Is pregnant or breastfeeding, or expecting to conceive or father children within the projected duration of the trial, starting with the pre-study visit through 120 days after the last dose of trial treatment; breastfeeding should be discontinued if the mother is treated with MK-3475 * Men and non-pregnant, non-breast-feeding women may be enrolled if they are willing to use 2 methods of birth control or are considered highly unlikely to conceive; highly unlikely to conceive is defined as 1) surgically sterilized, or 2) postmenopausal (a woman who is \>= 45 years of age and has not had menses for greater than 1 year will be considered postmenopausal), or 3) not heterosexually active for the duration of the study; the two birth control methods can be barrier method or a barrier method plus a hormonal method to prevent pregnancy; patients should start using birth control from the time of the pre-study visit, through the course of the study and for 120 days after the last dose of study medication; the following are considered adequate barrier methods of contraception: diaphragm, condom (by the partner), copper intrauterine device, sponge, or spermicide; appropriate hormonal contraceptives will include any registered and marketed contraceptive agent that contains an estrogen and/or a progestational agent (including oral, subcutaneous, intrauterine, or intramuscular agents) * Patients should be informed that taking the study medication may involve unknown risks to the fetus (unborn baby) if pregnancy were to occur during the study; in order to participate in the study they must adhere to the contraception requirement (described above) for the duration of the study and during the follow-up period; if there is any question that a patient will not reliably comply with the requirements for contraception, that patient should not be entered into the study * If a patient inadvertently becomes pregnant while on treatment with MK-3475, the patient will immediately be removed from the study; the site will contact the patient at least monthly and document the patient's status until the pregnancy has been completed or terminated; the outcome of the pregnancy will be reported without delay and within 24 hours if the outcome is a serious adverse experience (e.g., death, abortion, congenital anomaly, or other disabling or life-threatening complication to the mother or newborn); the study investigator will make every effort to obtain permission to follow the outcome of the pregnancy and report the condition of the fetus or newborn; if a male patient impregnates his female partner the study personnel at the site must be informed immediately and the pregnancy reported and followed * Patients who are human immunodeficiency virus (HIV) positive may participate IF they meet the following eligibility requirements: * They must be stable on their anti-retroviral regimen, and they must be healthy from an HIV perspective * They must have a CD4 count of greater than 250 cells/mcL * They must not be receiving prophylactic therapy for an opportunistic infection * Has known active hepatitis B (e.g., hepatitis B surface antigen \[HBsAg\] reactive) or hepatitis C (e.g., hepatitis C virus \[HCV\] ribonucleic acid \[RNA\] \[qualitative\] is detected) * Has received a live vaccine within 30 days prior to the first dose of trial treatment * Has a known human T-lymphotropic virus type 1 (HTLV) infection * Patient has a history of (non-infectious) pneumonitis that required steroids or current pneumonitis