IRX-2, Cyclophosphamide, and Pembrolizumab in Treating Participants With Recurrent or Metastatic Gastric or Gastroesophageal Junction Cancer

Inclusion Criteria: * Patients with histologically or cytologically confirmed recurrent or metastatic gastric or gastroesophageal junction (GEJ) adenocarcinoma progressed or intolerant to \>= 2 lines of systemic therapy. * Patients must have recurrent or metastatic gastric/GEJ adenocarcinoma that are not amenable to local therapy with curative intent (surgery or radiation therapy with or without chemotherapy). * Willing and able to give informed consent and adhere to protocol therapy; written informed consent and any locally required authorization must be obtained from the patient prior to performing any protocol-related procedures, including screening evaluations. * No prior exposure to PD-1/PD-L 1 inhibitor therapy. * Patients are deemed eligible for pembrolizumab therapy with tumors demonstrating PD-L1 expression by the Combined Positive Score (CPS) being \>= 1 as per the Food and Drug Administration (FDA)-approved Dako PD-L1 immunohistochemistry (IHC) 22C3 PharmDx assay. * Eastern Cooperative Oncology Group (ECOG) 0-1. * Body weight must be \> 30 Kg. * Hemoglobin \> 8 g/dL. * Absolute neutrophil count (ANC) \> 1,200 x 10\^9/mL. * Platelet count \> 75 x 10\^9/mL. * Serum bilirubin =\< 1.5 x institutional upper limit of normal (ULN) OR direct bilirubin =\< ULN if total bilirubin levels \> 1.5 x ULN. * Aspartate aminotransferase (AST/serum glutamic oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT/serum glutamate-pyruvate transaminase \[SGPT\]) =\< 5 x ULN. * Prothrombin time (PT) and partial thromboplastin time (PTT) \< 1.5 x the ULN. * Serum creatinine =\< 1.5 x ULN OR measured creatinine clearance (CL) \> 40 mL/min or calculated creatinine clearance CL \> 40 mL/min by the Cockcroft-Gault formula or by 24- hour urine collection for determination of creatinine clearance. * Palliative radiation therapy is allowed to non-target lesions at the discretion of the treating physician. * Patients must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded) or evaluable disease as outlined in Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1. * Life expectancy of greater than 3 months in the opinion of the treating physician. * Female patients of childbearing potential must have a negative serum pregnancy test within 7 days prior to enrollment. Exclusion Criteria: * Prior exposure to the IRX-2 regimen and/or PD-1/PD-L1 inhibitors are excluded. * Radiation therapy with a curable intent within 30 days of first dose of study treatment is excluded. However, radiation therapy with a palliative intent is allowed as long as treatment with study medication occurs \>= 14 days after the last dose of radiation. * Any medical contraindications or previous therapy that would preclude treatment with the IRX-2 regimen or pembrolizumab. * Known allergies to ciprofloxacin or phytohemagglutin given trace amount of these agents are contained in IRX-2. * Patients with ongoing chronic myelosuppression, myelodysplasia, or hemorrhagic cystitis which would contraindicate receipt of cyclophosphamide. * Any unresolved toxicity National Cancer Institute Common Terminology Criteria for Adverse Events (NCI CTCAE) grade \>= 2 from previous anticancer therapy with the exception of alopecia, vitiligo, and the laboratory values defined in the inclusion criteria. * Patients with grade \>= 2 neuropathy will be evaluated on a case-by-case basis after consultation with the study physician. * Patients with irreversible toxicity not reasonably expected to be exacerbated by treatment with IRX-2, pembrolizumab may be included only after consultation with the study physician. * Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease \[e.g., colitis or Crohn's disease\], diverticulitis \[with the exception of diverticulosis\], systemic lupus erythematosus, Sarcoidosis syndrome, or Wegener syndrome \[granulomatosis with polyangiitis, Graves' disease, rheumatoid arthritis, hypophysitis, uveitis, etc\]). The following are exceptions to this criterion: * Patients with vitiligo or alopecia. * Patients with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement. * Any chronic skin condition that does not require systemic therapy. * Patients without active disease in the last 2 years may be included but only after consultation with the study physician. * Patients with celiac disease controlled by diet alone. * Current or prior use of immunosuppressive medication within 14 days prior to cycle 1, day 1. The following are exceptions to this criterion: * Intranasal, inhaled, topical steroid or local steroid injections (e.g., intra articular injection). * Systemic corticosteroids at physiologic doses not to exceed 10 mg/day of prednisone or its equivalent. * Steroids as premedication for hypersensitivity reactions (e.g., computed tomography \[CT\] scan premedication). * Major surgical procedure (as defined by the Investigator) within 28 days prior to cycle 1, day 1. Note: Local surgery of isolated lesions for palliative intent is acceptable. * History of allogeneic organ transplantation. * Symptomatic cardiopulmonary disease (including congestive heart failure and hypertension), coronary artery disease, serious arrhythmia or chronic lung disease. Patients with these conditions who are stable with relatively minor symptoms and who are appropriate candidates for systemic treatments need not be excluded. * Myocardial infarction within the last 3 months. * Has a known history of human immunodeficiency virus (HIV) (HIV 1/2 antibodies). * Has a history of active hepatitis B requiring ongoing antiviral therapy or a history of untreated hepatitis C. * Has received a live vaccine within 4 months of planned start of study therapy (cycle 1, day 1). * Note: The killed virus vaccines used for seasonal influenza vaccines for injection are allowed provided not within 4 weeks of planned start of study therapy (cycle 1, day 1); however intranasal influenza vaccines (e.g., FluMist) are live attenuated vaccines and are not allowed. * Signs or symptoms of systemic infection (use of antibiotics to treat superficial infection or contamination of tumor shall not, by itself, be considered evidence of infection). * Stroke or other symptoms of cerebral vascular insufficiency within the last 3 months. * Previous diagnosis of invasive cancer from which the individual is not disease-free AND that has required treatment within the past 3 years, except for superficial skin, cervical cancer in-situ, or early stage prostate or bladder cancer (i.e. treatment with curative intent and long term disease-free expectations). * History of leptomeningeal carcinomatosis. * Known allergy or hypersensitivity to any of the study drugs or any of the study drug excipients. * Female patients who are pregnant or breastfeeding or male or female patients of reproductive potential who are not willing to employ effective birth control from screening to one year after last dose of cyclophosphamide or 180 days after the last dose of pembrolizumab therapy, whichever is longer.