Sorafenib Tosylate With or Without Stereotactic Body Radiation Therapy in Treating Patients With Liver Cancer

Inclusion Criteria: * Patients must have a diagnosis of HCC by at least one criterion listed below within 360 days prior to study entry: * Pathologically (histologically or cytologically) proven diagnosis of HCC,(biopsies are recommended, and are to be submitted for research evaluation if patients consent) * At least one solid liver lesion or vascular tumor thrombosis (involving portal vein, inferior vena cava (IVC) and/or hepatic vein) \> 1 cm with arterial enhancement and delayed washout on multi-phasic computerized tomography (CT) or magnetic resonance imaging (MRI) in the setting of cirrhosis or chronic hepatitis B or C without cirrhosis. * For patients whose CURRENT disease is vascular only: enhancing vascular thrombosis (involving portal vein, IVC and/or hepatic vein) demonstrating early arterial enhancement and delayed washout on multi-phasic CT or MRI in a patient with known HCC (diagnosed previously \<720 days) using the above criteria. * Measureable hepatic disease and/or presence of vascular tumor thrombosis (involving portal vein, IVC and/or hepatic vein) which may not be measureable as per Response Evaluation Criteria in Solid Tumors (RECIST) on liver CT or MRI, within 28 days of registration * Appropriate for protocol entry based upon the following minimum diagnostic workup: * History/physical examination including examination for encephalopathy, ascites, weight, height, and blood pressure within 14 days prior to study entry * Assessment by radiation oncologist and medical oncologist or hepatologist who specializes in treatment of HCC within 28 days prior to study entry * Pre-randomization Scan (REQUIRED for All Patients): Within 28 days prior to study entry, multiphasic liver CT or multiphasic liver MR scan. * Within 28 days prior to study entry CT chest with CT or MR abdomen and CT or MR pelvis, or positron emission tomography (PET) CT chest/abdomen/pelvis. * Zubrod performance status 0-2 within 28 days prior to study entry * All blood work obtained within 14 days prior to study entry with adequate organ marrow function defined as follows: * Absolute neutrophil count (ANC) \>= 1,500 cells/mm\^3 * Platelets \>= 60,000 cells/mm\^3 * Hemoglobin \>= 8.0 g/dl (note: the use of transfusion or other intervention to achieve hemoglobin \[Hgb\] \>= 8.0 g/dl is acceptable) * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) \< 6 times upper limit of normal (ULN) * Serum creatinine =\< 2 x ULN or creatinine clearance \>= 60 mL/min * Barcelona Clinic Liver Cancer (BCLC) stage: intermediate (B) or advanced (C) within 28 days prior to study entry * Child-Pugh score A within 14 days prior to study entry * Women of childbearing potential and male participants must agree to practice adequate contraception while on study and for at least 6 months following the last dose of radiation therapy (RT) and for at least 28 days following the last dose of sorafenib (whichever is later) * Unsuitable for resection or transplant or radiofrequency ablation (RFA) * Unsuitable for or refractory to transarterial hepatic chemo-embolization (TACE) or drug eluting beads (DEB) for any of the following reasons, as described by Raoul et al (2011): * Technical contraindications: arteriovenous fistula, including, surgical portosystemic shunt or spontaneous portosystemic shunt * Severe reduction in portal vein flow: due to tumor portal vein, IVC or atrial invasion or bland portal vein occlusion * Medical contraindications including congestive heart failure, angina, severe peripheral vascular disease * Presence of extrahepatic disease * No response post TACE (or DEB) or progressive HCC despite TACE; prior TACE or DEB is allowed but must be \> 28 days from study entry * Serious toxicity following prior TACE (or DEB); prior TACE or DEB must be \> 28 days from study entry * Other medical comorbidities making TACE (or DEB) unsafe and/or risky (e.g. combination of relative contraindications including age \> 80 years, tumor \> 10 cm, \> 50% replacement of the liver by HCC, extensive multinodular bilobar HCC, biliary drainage) * Patients treated with prior surgery are eligible for this study if they otherwise meet eligibility criteria * Patient must be able to provide study-specific informed consent prior to study entry Exclusion Criteria: * Prior invasive malignancy (except non-melanomatous skin cancer and T1 renal cell carcinoma) unless disease free for a minimum of 2 years (note that carcinoma in situ of the breast, oral cavity, or cervix are all permissible) * Prior sorafenib use \> 60 days and/or grade 3 or 4 sorafenib related toxicity. Note that prior chemotherapy for HCC or a different cancer is allowable * Prior radiotherapy to the region of the liver that would result in overlap of radiation therapy fields * Prior selective internal radiotherapy/hepatic arterial yttrium therapy, at any time * Severe, active co-morbidity, defined as follows: * Unstable angina and/or congestive heart failure requiring hospitalization within the last 6 months PRIOR TO registration * Transmural myocardial infarction within the last 6 months prior to study entry * Unstable ventricular arrhythmia within the last 6 months prior to study entry * Acute bacterial or fungal infection requiring intravenous antibiotics within 28 days prior to study entry * Hepatic insufficiency resulting in clinical jaundice, encephalopathy and/or variceal bleed within 28 days prior to study entry * Bleeding within 28 days prior to study entry due to any cause, requiring transfusion * Thrombolytic therapy within 28 days prior to study entry. Subcutaneous heparin is permitted. * Known bleeding or clotting disorder * Uncontrolled psychotic disorder * Pregnancy or women of childbearing potential and men who are sexually active and not willing/able to use medically acceptable forms of contraception; this exclusion is necessary because the treatment involved in this study may be significantly teratogenic * Maximal diameter of any one hepatocellular carcinoma \> 15 cm * Total sum of maximum diameters of each definite parenchymal hepatocellular carcinoma within the liver or maximum diameter of a single conglomerate HCC \> 20 cm * More than 5 discrete intrahepatic parenchymal foci of HCC * Direct tumor extension into the stomach, duodenum, small bowel or large bowel * Measureable common or main branch biliary duct involvement with HCC * Extrahepatic metastases or malignant nodes (that enhance with typical features of HCC) \> 3.0 cm, in sum of maximal diameters (e.g. presence of one 3.4 cm metastatic lymph node or two 2 cm lung lesions); note that benign non-enhancing periportal lymphadenopathy is not unusual in the presence of hepatitis and is permitted, even if the sum of enlarged nodes is \> 2.0 cm * Prior liver transplant * HIV positive with CD4 (T-cell count) count \< (350) cells/microliter. Note that patients who are HIV positive are eligible, provided they are under treatment with highly active antiretroviral therapy (HAART) and have a CD4 count ≥ (350) cells/microliter, and no known detectable viral load, at the time of study entry. Note also that HIV testing is not required for eligibility for this protocol