Sorafenib Tosylate and Pembrolizumab in Treating Patients With Advanced or Metastatic Liver Cancer

Inclusion Criteria: * Participant must have histologically or radiographically confirmed hepatocellular cancer (HCC) that is advanced or metastatic and if archival tissue is available, have archival tissue submitted for PD-L1, PD-L2 testing * Participants with measurable disease that has progressed are eligible if prior surgery or locoregional therapy occurred \> 28 days prior to enrollment * Have an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1 (Karnofsky \>= 60%) * Child-Pugh class-A liver function * Absolute neutrophil count (ANC) \>= 1,500/ mcL * Hemoglobin \>= 8.5 g/dL * Platelets \>= 75,000/ mcL * Total bilirubin =\< 2.0 mg/dL * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) =\< 5 X ULN * Serum Creatinine \<= 1.5 upper limit of normal (ULN)or Creatinine clearance \> 50 mL/minute if serum creatinine is elevated above 1.5 X ULN * Have measurable disease per Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria present * Ability to swallow and retain oral medication * Participants of child-bearing potential must agree to use adequate contraceptive methods (e.g., hormonal or barrier method of birth control; abstinence) prior to study entry; should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately * Participant must understand the investigational nature of this study and sign an Independent Ethics Committee/Institutional Review Board approved written informed consent form prior to receiving any study related procedure * Participants with past or ongoing hepatitis C virus (HCV) infection will be eligible for the study. The treated participants must have completed their treatment at least 1 month prior to starting study intervention. * Participants with controlled hepatitis B will be eligible as long as they meet the following criteria: Antiviral therapy for HBV must be given for at least 12 weeks and HBV viral load must be less than 100 IU/ml prior to first dose of study drug. Participants on active HBV therapy with viral loads under 100 IU/mL should stay on the same therapy throughout study treatment Participants who are anti-HBc , negative for Hepatitis B surface antigen (HBsAg) and negative or positive for anti-HBs, and who have an HBV viral load under 100 IU/mL , do not require HBV anti-viral prophylaxis Exclusion Criteria: * One prior line of therapy that may include a PDL1 blocker allowed, no prior sorafenib or PD1 blocker allowed. * Participants who have had radiotherapy or chemotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agents administered more than 4 weeks earlier * Any evidence of bleeding diathesis (patients on therapeutic warfarin or heparin will be excluded) * Participants with a history of variceal bleed within 6 months prior to enrollment * Known human immunodeficiency virus (HIV)-positive participants (even if on combination retrovirals, participant will be excluded * Participants with chronic autoimmune disease * Participants with known brain metastases should be excluded from this clinical trial * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Has known history of, or any evidence of active, non-infectious pneumonitis * Pregnant or nursing female participants * Unwilling or unable to follow protocol requirements * Any condition which in the Investigator's opinion deems the participant an unsuitable candidate to receive study drug * Received a live vaccine within 30 days prior to start of study treatment