Preoperative Immunotherapy (Pembrolizumab) for Patients With Colorectal Cancer and Resectable Hepatic Metastases

Inclusion Criteria: 1. Has histologically or cytologically confirmed CRC with liver metastases. In addition to liver metastases, extrahepatic metastases (e.g. pulmonary metastases) may be permitted if all other eligibility criteria are met. Participants are permitted to have primary tumor in situ (Neoadjuvant Arm only). 2. Has received previous oxaliplatin-based chemotherapy. a. FOLFOX or capecitabine combined with oxaliplatin (CapeOx) does not need to be a direct lead-in to this study. b. If chemotherapy is a direct lead-in to this study, concurrent mAb therapy (bevacizumab, cetuximab, or panitumumab) is acceptable, however the antibody must be omitted from the final cycle of chemotherapy prior to pembrolizumab. 3. Is an appropriate candidate to undergo liver biopsy and resection (+/-ablation) of liver metastases (Neoadjuvant Arm Only). 4. Is willing and able to provide written informed consent/assent for the trial. The participants may also provide consent for Future Biomedical Research. However, may participate in the main trial without participating in Future Biomedical Research. 5. Is \>=18 years of age on day of signing informed consent. 6. Has measurable disease based on Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 as assessed by the investigator. Lesions situated in a previously irradiated area are considered measurable if progression has been demonstrated in such lesions (Neoadjuvant Arm Only). 7. Is willing to provide tissue from a newly obtained core or excisional biopsy of a tumor lesion. Newly-obtained is defined as a specimen obtained after last dose of standard of care lead-in chemotherapy \[if applicable\] and within 28 days prior to first dose of pembrolizumab (Neoadjuvant Arm Only). 8. Has a performance status of 0 or 1 on the Eastern Cooperative Oncology Group (ECOG) Performance Scale. 9. Has adequate organ function within 10 days of treatment initiation. \- Absolute neutrophil count (ANC) \>= 1,500/microliter (uL) (within 10 days of treatment initiation). * Platelets \>= 100,000/uL (within 10 days of treatment initiation). * Hemoglobin \>= 9 g/dL or \>= 5.6 mmol/L without transfusion or erythropoietin (EPO) dependency (within 7 days of assessment). * Serum creatinine =\< 1.5 x upper limit of normal (ULN) OR measured or calculated creatinine clearance (glomerular filtration rate (GFR) can also be used in place of creatinine or creatinine clearance (CrCl)) \>= 60 mL/min for participants with creatinine levels \> 1.5 x institutional ULN (within 10 days of treatment initiation). * Creatinine clearance should be calculated per institutional standard. * Serum total bilirubin =\< 1.5 x ULN OR direct bilirubin =\< ULN for participants with total bilirubin levels \> 1.5 ULN (within 10 days of treatment initiation). * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase (SGOT) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase (SGPT)) =\< 2.5 x ULN (within 10 days of treatment initiation). * Albumin \>= 2.5 mg/dL (within 10 days of treatment initiation). * International normalized ratio (INR) or prothrombin time (PT) =\< 1.5 x ULN unless participant is receiving anticoagulant therapy as long as PT or partial thromboplastin time (PTT) is within therapeutic range of intended use of anticoagulants (within 10 days of treatment initiation). * Activated partial thromboplastin time (aPTT) =\< 1.5 x ULN unless participants is receiving anticoagulant therapy as long as PT or PTT is within therapeutic range of intended use of anticoagulants (within 10 days of treatment initiation). 10. Female participants of childbearing potential must have a negative urine or serum pregnancy test within 72 hours prior to receiving the first dose of study medication (day 1). If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required. 11. Female participants of childbearing potential must be willing to use an adequate method of contraception for the course of the study through 120 days after the last dose of study medication. 12. Male participants of childbearing potential must agree to use an adequate method of contraception, starting with the first dose of study therapy through 120 days after the last dose of study therapy. \*Note: Abstinence is acceptable if this is the usual lifestyle and preferred contraception for the participants. 13. Have locally confirmed microsatellite stable (MSS) or Mismatch repair proficient (pMMR) Colorectal cancer (CRC). MSS is defined as 0-1 allelic shifts among 3-5 tumor microsatellite loci using a Polymerase chain reaction (PCR)-based assay. pMMR is defined as presence of protein expression of 4 DNA mismatch repair (MMR) enzymes (MLH1, MSH2, MSH6 and PMS2) by immunohistochemistry. 14. Have no radiographic evidence of malignancy as assessed by Investigator (Adjuvant Phase Only). Exclusion Criteria: 1. Is currently participating and receiving study therapy or has participated in a study of an investigational agent and received study therapy or used an investigational device within 4 weeks of the first dose of treatment. * Note: Participants who have entered the follow-up phase of an investigational study may participate as long as it has been 4 weeks since the last dose of the previous investigational agent or device use. 2. Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of trial treatment. 3. Has a known history of human immunodeficiency virus (HIV) infection. No HIV testing is required unless mandated by local health authority. 4. Has an active autoimmune disease that has required systemic treatment in past 2 years (i.e., with use of disease-modifying agents, corticosteroids, or immunosuppressive drugs). Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency) is not considered a form of systemic treatment and is allowed. 5. Has active hepatitis B (defined as hepatitis B surface antigen (HBsAg) reactive) or active hepatitis C virus (HCV) (defined as HCV RNA \[qualitative\] is detected) infection. 6. Has received prior anti-cancer monoclonal antibody (mAb), systemic anticancer therapy other than FOLFOX (including investigational agents), targeted small molecule therapy, or radiation therapy within 14 days prior to the first dose of study treatment (day 1). * Note: Participants must have recovered from all adverse events (AEs) due to a previous therapies to =\< grade 1 or baseline. Participant with grade 2 neuropathy or alopecia are eligible. If a participant received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting study treatment. 7. Has received FOLFOX less than 7 days prior to the first dose of study treatment (day 1). Has not recovered (i.e., =\< grade 1 or at baseline) from AEs due to FOLFOX chemotherapy. * Note: Participants with =\< grade 2 neuropathy or alopecia are exceptions to this criterion and may qualify for the study. 8. Has not recovered adequately from toxicity or complications of a surgery or other procedure, per the assessment of the treating investigator. 9. Has received liver-directed therapy such as radiotherapy or yttrium-90 in the past year involving the lesion to be biopsied. (Neoadjuvant Arm only) 10. Has a known additional malignancy that is progressing or has required active treatment within 5 years prior. * Note: Participants with basal cell carcinoma of the skin, squamous cell carcinoma of the skin, carcinoma in situ (e.g. breast carcinoma, cervical cancer in situ) that have undergone potentially curative therapy are not excluded. 11. Has known active central nervous system (CNS) metastases and/or carcinomatous meningitis. Participants with previously treated brain metastases may participate provided they are radiologically stable (i.e., without evidence of progression by imaging for at least 4 weeks by repeat imaging \[repeat imaging should be performed during the study screening\]), clinically stable, and without requirement of steroid treatment for at least 14 days prior to trial treatment. This exception does not include carcinomatous meningitis which is excluded regardless of clinical stability. 12. Has a history of (noninfectious) pneumonitis that required steroids or has current pneumonitis. 13. Has an active infection requiring systemic therapy. 14. Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with participation for the full duration of the trial, or is not in the best interest of the participant, in the opinion of the treating investigator. Anticoagulation that cannot be safely held to perform the liver biopsy is an example of a contraindication to participation. 15. Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. Has received any prior immunotherapy including anti-PD-1, anti-PD-L1, or anti-PD-L2 or with an agent directed to another stimulatory or co-inhibitory T-cell receptor (e.g. cytotoxic T-lymphocyte-associated protein 4 (CTLA-4), OX-40, CD137). (Neoadjuvant Arm Only). 17\. Has severe hypersensitivity (\>= grade 3) to pembrolizumab and/or to any of pembrolizumab's excipients. 18\. Has received a live or live-attenuated vaccine within 30 days prior to first dose of the trial drug. Administration of killed vaccines are allowed. 19\. Has inferior vena cava/cardiac involvement based on imaging. 20. Has had encephalopathy in the last 6 months. Those participants on rifaximin or lactulose to control their encephalopathy are not allowed. 21\. Has had a solid organ or hematologic transplant. 22. Has symptomatic ascites or pleural effusion. A participant who is clinically stable following treatment for these conditions (including therapeutic thora- or paracentesis) is eligible. 23\. Uncontrolled intercurrent illness, including but not limited to, ongoing or active infection, symptomatic congestive heart failure (New York Heart Association Class III/IV), uncontrolled hypertension (\>=150/90mmHg), unstable angina pectoris or myocardial infarction (≤ 6 months prior to screening), uncontrolled cardiac arrhythmia, clinically significant cardiac valvulopathy requiting treatment, active interstitial lung disease, or serious chronic gastrointestinal conditions associated with diarrhea.