Pembrolizumab in Treating Patients With Metastatic or Unresectable Gastroesophageal Adenocarcinoma

Inclusion Criteria: * The subject (or legally acceptable representative if applicable) provides written informed consent for the trial. The subject may also provide consent for future biomedical research. However the subject may participate in the main trial without participating in future biomedical research. * Be willing and able to provide written informed consent/assent for the trial. * Histologically and cytologically documented diagnosis as gastroesophageal adenocarcinoma. * Have a documented advanced (metastatic and/or unresectable) gastroesophageal adenocarcinoma that is incurable and for which prior first-line or later-line standard of care (SOC) treatments have failed. There is no limit to the number of prior treatment regimens. Prior neoadjuvant or adjuvant therapy included in initial treatment may not be considered first- or later-line SOC treatment unless such treatments were completed less than 12 months prior to the current tumor recurrence. * Have submitted an evaluable tissue sample for biomarker analysis from a newly obtained endoscopic, core, incisional, or excisional biopsy of a tumor lesion not previously irradiated. The tumor tissue submitted for analysis must be from a single tumor tissue specimen and of sufficient quantity and quality to allow biomarker study. A newly obtained tumor specimen, defined as a specimen obtained up to 6 weeks (42 days) prior to initiation of treatment on day 1, for biomarker characterization will be required for enrollment of all subjects. Tissue from tumor progressing at a site of prior radiation may be allowed for biomarker characterization upon agreement from Merck. Subjects for whom newly-obtained samples cannot be provided (e.g. inaccessible or subject safety concern) may submit an archived specimen only upon agreement from the Merck. * Have measurable disease based on RECIST 1.1 as assessed by the investigator. Lesions situated in a previously irradiated area are considered measurable if progression has been demonstrated in such lesions. * Have a performance status of 0 or 1 on the Eastern Cooperative Oncology Group (ECOG) performance scale. * Life expectancy of greater than 3 months per the judgment of the investigators. * Absolute neutrophil count (ANC) \>= 1,500 /mcL (within 5 days prior to the start of study treatment). * Platelets \>= 100,000/mcL (within 5 days prior to the start of study treatment). * Hemoglobin \>= 9 g/dL or \>= 5.6 mmol/L without transfusion or erythropoietin (EPO) dependency (within 7 days of assessment) (within 5 days prior to the start of study treatment). * Serum creatinine OR measured or calculated creatinine clearance (glomerular filtration rate \[GFR\] can also be used in place of creatinine or creatinine clearance \[CrCl\]) =\< 1.5 X upper limit of normal (ULN) OR \>= 60 mL/min for subject with creatinine levels \> 1.5 X institutional upper limit of normal (ULN) (within 5 days prior to the start of study treatment). Creatinine clearance should be calculated per institutional standard. * Serum total bilirubin =\< 1.5 X ULN OR direct bilirubin =\< ULN for subjects with total bilirubin levels \> 1.5 ULN (within 5 days prior to the start of study treatment). * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 X ULN OR =\< 5 X ULN for subjects with liver metastases (within 5 days prior to the start of study treatment). * Albumin: \>= 2.5 mg/dL (within 5 days prior to the start of study treatment). * International normalized ratio (INR) or prothrombin time (PT) =\< 1.5 X ULN unless subject is receiving anticoagulant therapy as long as PT or partial thromboplastin time (PTT) is within therapeutic range of intended use of anticoagulants (within 5 days prior to the start of study treatment). * Activated partial thromboplastin time (aPTT) =\< 1.5 X ULN unless subject is receiving anticoagulant therapy as long as PT or PTT is within therapeutic range of intended use of anticoagulants (within 5 days prior to the start of study treatment). * Female subject of childbearing potential should have a negative urine or serum pregnancy within 72 hours prior to receiving the first dose of study medication. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required. * Male and female subjects of childbearing potential must be willing to use an adequate method of contraception, for the course of the study through 120 days after the last dose of study medication. Note: Abstinence is acceptable if this is the usual lifestyle and preferred contraception for the subject. Exclusion Criteria: * Is currently participating in or has participated in a study of an investigational agent or used an investigational device within 4 weeks prior to the first dose of study treatment. Note: Subjects who have entered the follow-up phase of an investigational trial may participate as long as it has been 4 weeks after the last dose of the previous investigational agent. * Has a diagnosis of immunodeficiency or is receiving systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of trial treatment. * Has severe hypersensitivity (\>= grade 3) to pembrolizumab or any of its excipients. * Has had a prior anti-cancer monoclonal antibody (mAb) within 4 weeks prior to study day 1 or who has not recovered (i.e., =\< grade 1 or at baseline) from adverse events due to agents administered more than 4 weeks earlier. * Has had prior chemotherapy, targeted small molecule therapy, or radiation therapy within 2 weeks prior to study day 1 or who has not recovered (i.e., =\< grade 1 or at baseline) from adverse events due to a previously administered agent. * Note: Subjects with =\< grade 2 neuropathy or alopecia are an exception to this criterion. * Note: If subject received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention as determined by the investigators prior to starting therapy. * Has a known additional malignancy that is progressing or requires active treatment. Note: Subjects with basal cell carcinoma of the skin, squamous cell carcinoma of the skin, or carcinoma in situ (e.g., breast carcinoma, cervical cancer in situ) that have undergone potentially curative therapy are not excluded. * Has known active central nervous system (CNS) metastases and/or carcinomatous meningitis. Subjects with previously treated brain metastases may participate provided they are radiological stable (i.e., without evidence of progression for at least 4 weeks by repeat imaging (note that the repeat imaging should be performed during study screening), clinically stable and without requirement of steroid treatment for at least 14 days prior to the first dose of trial treatment. This exception does not include carcinomatous meningitis which is excluded regardless of clinical stability. * Has an active autoimmune disease that has required systemic treatment in the past 2 years (i.e., with use of disease modifying agents, corticosteroids or immunosuppressive drugs). Replacement therapy (e.g. thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment and is allowed. * Has a history of (non-infectious) pneumonitis that required treatment with steroids or has current pneumonitis. * Has an active infection requiring systemic therapy. * Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the subject's participation for the full duration of the trial, or is not in the best interest of the subject to participate, in the opinion of the treating investigator. * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. * Is pregnant or breastfeeding, or expecting to conceive or father children within the projected duration of the trial, starting with the visit through 120 days after the last dose of trial treatment. * Has received prior therapy with an anti-PD-1, anti-PD-L1, or anti-PD-L2 agent or with an agent directed to another stimulatory or co-inhibitory T-cell receptor (TCR) (e.g., CTLA-4, OX 40, CD137). * Has a known history of human immunodeficiency virus (HIV) (HIV 1/2 antibodies). No HIV testing is required unless mandated by local health authority. * Had a solid organ or hematologic transplant. * Has a known history of hepatitis B virus (HBV; hepatitis B surface antigen \[HBsAg\] reactive) or known active hepatitis C virus (HCV; HCV ribonucleic acid \[RNA\] \[qualitative\] is detected) infection. No testing for hepatitis B and hepatitis C is required unless mandated by local health authority. * Has received a live vaccine within 30 days prior to the first dose of the trial drug. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, varicella/zoster (chicken pox), yellow fever, rabies, Bacillus Calmette - Guerin (BCG), and typhoid vaccine.