Combined Treatment of Durvalumab, Bevacizumab, Tremelimumab and Transarterial Chemoembolization (TACE) in Subjects With Hepatocellular Carcinoma or Biliary Tract Carcinoma

* INCLUSION CRITERIA: * Participants must have * histopathological confirmation of HCC (Cohorts 1 and 3) OR * histopathological confirmation of BTC or histopathological confirmation of carcinoma in the setting of clinical and radiological characteristics which, together with the pathology, are highly suggestive of a diagnosis of BTC (Cohort 2). * Participants should have progressed on standard of care systemic therapy or been intolerant of or have refused standard treatment. Note: For participants enrolled in Cohort 3 (HCC, BCLC stage B), standard of care chemotherapy is not required prior to enrollment. * Participants must have disease that is not amenable to potentially curative resection, radiofrequency ablation, or liver transplantation * Participants must have evaluable or measurable disease per RECIST 1.1 * Participants must have at least one lesion accessible for TACE (Cohort 3) * Participants must have lesions accessible for biopsy and be willing to undergo pre- and posttreatment biopsies * ECOG performance status of 0 to 1 * If liver cirrhosis is present, patient must have a Child-Pugh score \<7 * Subjects with HCC must have BCLC C (Cohort 1) or BCLC B (Cohort 3) * Participants must have normal organ and marrow function as defined below: * absolute neutrophil count greater than or equal to 1,000/mcL * platelets greater than or equal to 60,000/mcL * total bilirubin:if cirrhosis present: Part of Child Pugh requirement-If no cirrhosis: bilirubin should be less than or equal to 2 XULN * ALT or AST up to 5 x ULN * Creatinine OR measured or calcutated Creatinine clearance (crCl) (eGFR may Also be used in place of CrCl) A: less than the institutional limit of normal OR greater than or equal to 45/mL/1.73 m\^2 for participant with creatinine levels greater than or equal to 1.5 X institutional ULN * No proteinuria: Urine dipstick \<2. Participants discovered to have greater than or equal to 2 + proteinuria on dipstick analysis should undergo a 24-hour urine collection and must demonstrate less than or equal to 1g of protein in 24 hours to be eligible * Age greater than or equal to 18 years * Participants must have recovered from any acute toxicity related to prior therapy, including surgery. Toxicity should be less than or equal to grade 1. * The effects of study drugs on the developing human fetus are unknown. For this reason, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) at the study entry and for the duration of study treatment and up to 90 days after the last dose of the study drug(s). Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. * HBV infected subjects must be on antivirals and have HBV DNA \<100IU/mL. HCV infected subjects can be enrolled with close HCV RNA level monitoring * Body weight \>30kg * Participant must be able to understand and willing to sign a written informed consent document. EXCLUSION CRITERIA: * Participants who have had standard-of-care anti-cancer therapy or therapy with investigational agents (e.g. chemotherapy, endocrine therapy, targeted therapy, biologic therapy, tumor embolization, monoclonal antibodies or other investigation agents) or large field radiotherapy within 4 weeks prior to treatment initiation. * Major surgery within 6 weeks prior to treatment initiation. Minor procedures (e.g. port placement, endoscopy with intervention) within 2 weeks prior to treatment initiation. * Active central nervous system metastases and/or carcinomatous meningitis. Particpants with known active brain metastases will be excluded from this clinical trial because of their poor prognosis and because they often develop progressive neurologic dysfunction that would confound the evaluation of neurologic and other adverse events * Metastatic disease that involves major airways or blood vessels, or centrally located mediastinal tumor masses. * Medical condition that requires chronic systemic steroid therapy, or any other form of immunosuppressive medication (inhaled and topical steroids are permitted). * Chronic daily treatment with a non-steroidal anti-inflammatory drug (NSAID). * A prior bleeding event due to esophageal and/or gastric varices within 6 months prior to initiation of study treatment. * Inadequately controlled arterial hypertension (defined as systolic blood pressure (BP) \> 150 mmHg and/or diastolic blood pressure \> 100 mmHg), based on an average of 3 BP readings on 2 sessions. Note: anti-hypertensive therapy to achieve these parameters is allowable. * Prior history of hypertensive crisis or hypertensive encephalopathy * Significant vascular disease (e.g., aortic aneurysm requiring surgical repair or recent peripheral arterial thrombosis) within 6 months prior to initiation of study treatment * Evidence of bleeding diathesis or significant coagulopathy (with or without current therapeutic anticoagulation). * Recent (within 10 days of first dose of study treatment) use of aspirin * Thromboembolic event within 6 months of initiation of study treatment (including cerebrovascular accident (CVA) and myocardial infarction (MI). * History of hemoptysis (\>2.5 mL of bright red blood per episode) within 1 month prior to treatment initiation. * Serious, non-healing wound, active ulcer, or untreated bone fracture. * HIV-positive participants are excluded because HIV causes complicated immune deficiency and study treatment can pose more risks for these patients. * History of severe hypersensitivity reaction to any monoclonal antibody. * Congestive heart failure, transmural myocardial infarction, angina pectoris requiring medication, clinically significant valvular disease, high-risk arrhythmia within 12 months prior to treatment initiation. Prior history of hypertensive crisis or hypertensive encephalopathy. * Prior invasive malignancies within the past 5 years prior to treatment initiation (with the exception of non-melanoma skin cancers, non-invasive bladder cancer or localized prostate cancer for whom systemic therapy is not required) * Active or history of inflammatory bowel disease (colitis, Crohn s), irritable bowel disease, celiac disease, or other serious, chronic, gastrointestinal conditions associated with diarrhea. * History of abdominal fistula or gastrointestinal perforation within 6 months prior to initiation of study treatment. * History of chronic autoimmune disease (e.g., Addison s disease, multiple sclerosis, Graves disease, Hashimoto s thyroiditis, rheumatoid arthritis, hypophysitis, systemic lupus erythematosus, Wegener s granulomatosis, sarcoidosis syndrome etc.) or other connective tissue diseases with symptomatic disease within the 3 years of initiation of study treatment. Note: Active vitiligo or a history of vitiligo will not be a basis for exclusion. * Diverticulitis either active or history of within 2 years of initiation of study treatment. Note that diverticulosis is permitted. * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection or psychiatric illness/social situations that may impair the patient s tolerance of study treatments. * Received any live vaccine within the last 30 days before treatment initiation. * Participants who have undergone prior liver transplantation. * Pregnant women are excluded from this study because durvalumab s and bevacizumab s potential for teratogenic or abortifacient effects is unknown. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with tremelimumab, durvalumab and bevacizumab, breastfeeding should be discontinued if the mother is treated with study drugs.