Pembrolizumab in Treating Patients With Stage IV Metastatic or Recurrent Inflammatory Breast Cancer or Triple-Negative Breast Cancer Who Have Achieved Clinical Response or Stable Disease to Prior Chemotherapy

Inclusion Criteria: * Is willing and able to provide written informed consent for the trial * Is a female or male and \>/= 18 years of age * Has histological confirmation of HER2 normal breast carcinoma with a clinical diagnosis of IBC based on presence of inflammatory changes in the involved breast, including diffuse erythema and edema (peau d'orange), with or without an underlying palpable mass involving the majority of the skin of the breast; pathological evidence of dermal lymphatic invasion should be noted but is not required for diagnosis of inflammatory breast cancer regardless estrogen receptor (ER)/progesterone receptor (PR) status; OR has histological confirmation of triple negative breast carcinoma (HER2 normal, ER/PR \< 10%) without clinical diagnosis of IBC * Has stage IV or recurrent disease that has been treated * Has clinical response or stable disease for minimum of two months (three cycles of every three week chemotherapy or 8 weeks of weekly regimen, etc.) after receiving any prior chemotherapy for metastatic/recurrent disease; a minimum of two cycles (6-8 weeks) of chemotherapy is required to determine clinical response. Per RECIST criteria 1.1, Clinical response for measurable disease is defined as complete response (CR) or partial response (PR); for non-measurable disease only (i.e. bone metastasis, ascites, pleural effusion, and pathological lymph nodes \>/= 10 to \<15 mm short axis) is defined as persistence of one or more non-target lesion(s) and no increase in overall tumor burden. * Is HER2 normal, defined as HER2 0 or 1+ by IHC and negative by FISH if performed; or HER2 is 2+ by IHC and negative by FISH; or HER2 negative by FISH if IHC is not performed. * Has a performance status of 0-1 on the Eastern Cooperative Oncology Group (ECOG) performance scale * Has adequate organ function as determined by the following laboratory values: ANC \>/= 1000 /mcL, Platelets \>/=100,000 /mcL, Hgb \>/= 9 g/dL, creatinine levels \< 1.5 x ULN, Total bilirubin \</= 1.5 x ULN, ALT and AST \</= 2.5 x ULN or \</=5 x ULN for participants with liver metastases. * Participants of childbearing potential should be willing to use effective methods of birth control or be surgically sterile, or abstain from heterosexual activity for the course of the study through at least 4 months after the last dose of study drug. Participants of childbearing potential are those who have not been surgically sterilized or have not been free from menses for \> 1 year. Effective methods of birth control include 1). Use of hormonal birth control methods: pills, shots/injections, implants (placed under the skin by a health care provider), or patches (placed on the skin); 2).Intrauterine devices (IUDs); 3).Using 2 barrier methods (each partner must use 1 barrier method) with a spermicide. Males must use the male condom (latex or other synthetic material) with spermicide. Females must choose either a Diaphragm with spermicide, or Cervical cap with spermicide, or a sponge (spermicide is already in the contraceptive sponge). * Has negative serum or urine pregnancy test for participants of childbearing potential Exclusion Criteria: * Is currently participating in a study of an investigational anti-cancer agent * Has a diagnosis of immunodeficiency or is receiving systemic steroid therapy or any other form of immunosuppressive therapy * Has not recovered from adverse events due to prior therapies, i.e. monoclonal antibody, chemotherapy, targeted small molecule therapy, radiation therapy, or surgery. (Note: Participants with ≤ grade 2 neuropathy, alopecia and general disorders and administration site conditions \[per Common Terminology Criteria for Adverse Events (CTCAE version 4.0) are an exception to this criterion and may qualify for the study) * Has a known malignancy (other than breast cancer) except basal cell carcinoma or squamous cell carcinoma of the skin, or in situ cervical cancer that has undergone potentially curative therapy * Has known active central nervous system (CNS) metastases and/or carcinomatous meningitis; participants with previously treated brain metastases may participate if they are stable, and have no evidence of new or enlarging brain metastases, and are not using steroids for at least 7 days prior to trial treatment * Has an active autoimmune disease requiring systemic treatment within the past 3 months or a documented history of clinically severe autoimmune disease, or a syndrome that requires systemic steroids or immunosuppressive agents; participants with vitiligo or resolved childhood asthma/atopy would be an exception to this rule. Participants that require intermittent use of bronchodilators or local steroid injections would not be excluded from the study. Participants with hypothyroidism stable on hormone replacement or Sjogren's syndrome will not be excluded from the study. * Has a history of (non-infectious) pneumonitis that required steroids or current pneumonitis * Has an active infection requiring systemic therapy * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial * Has received prior therapy with an anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CD137, or anti-Cytotoxic T-lymphocyte-associated antigen-4 (CTLA-4) antibody (including ipilimumab or any other antibody or drug specifically targeting T-cell co-stimulation or checkpoint pathways). * Has a known history of human immunodeficiency virus (HIV) * Has a known active hepatitis B or hepatitis C * Have received a live vaccine within 30 days prior to the first dose of trial treatment * Is receiving concurrent anti-cancer therapy for metastatic disease