Study of Nivolumab in Combination With Gemcitabine/Cisplatin or Ipilimumab for Patients With Advanced Unresectable Biliary Tract Cancer

Inclusion Criteria: * Patients must have a pathologically confirmed adenocarcinoma of the biliary tract (intra-hepatic, extra-hepatic (hilar, distal) or gall bladder) that is not eligible for curative resection, transplantation, or ablative therapies. Tumors of mixed histology are excluded. * Patients may have received prior radiation, chemoembolization, radioembolization or other local ablative therapies or hepatic resection if completed ≥ 4 weeks prior to registration AND if patient has recovered to \<= grade 1 toxicity. Extrahepatic palliative radiation is permitted if completed ≥ 2 weeks prior to enrollment AND if patient has recovered to ≤ grade 1 toxicity. * Patients must have radiographically measurable disease in at least one site not previously treated with radiation or liver directed therapy (including bland, chemo- or radio-embolization, or ablation) either within the liver or in a metastatic site. * Must be ≥18 years of age * Must have a Child-Pugh score of A (prognosis in chronic liver disease and cirrhosis) * Must have an ECOG (Eastern Cooperative Oncology Group) performance status of 0-1 * Ability to understand and willingness to sign IRB-approved informed consent * Willing to provide archived tissue, if available, from a previous diagnostic biopsy * Must be able to tolerate CT (computerized tomography) and/or MRI (magnetic resonance imaging) with contrast * Must have adequate organ function obtained ≤ 2 weeks prior to registration Exclusion Criteria: * Patients may not have received prior systemic treatment (chemotherapy or targeted therapy) for advanced BTC (biliary tract cancer). Prior adjuvant chemotherapy is permitted provided it was completed \> 6 months from registration. * Must not have a diagnosis of immunodeficiency, or have received systemic steroid therapy, or any other form of immunosuppressive therapy within 7 days prior to trial treatment. * Must not have known Hepatitis B, Hepatitis C, or HIV seropositivity. Testing is not required in absence of clinical suspicion. * Must not have prior history of organ transplantation or brain metastasis. * Must not have undergone a major surgical procedure \< 4 weeks prior to registration. * Must not have an active second malignancy other than non-melanoma skin cancer or cervical carcinoma in situ. Patients with history of malignancy are eligible provided primary treatment of that cancer was completed \> 1 year prior to registration and the patient is free of clinical or radiologic evidence of recurrent or progressive malignancy. * Must have no ongoing active, uncontrolled infections * Must not have received a live vaccine within 30 days of planned start of the study therapy. * Must not have a psychiatric illness, other significant medical illness, or social situation which, in the investigator's opinion, would limit compliance or ability to comply with study requirements. * Women must not be pregnant or breastfeeding since study drugs may harm the fetus or child. * Women of child-bearing potential and men must agree to use 2 methods of adequate contraception (hormonal plus barrier or 2 barrier forms) OR abstinence prior to study entry, for the duration of study participation and for 5 months (for women) and 7 months (for men) following completion of study therapy. * Participants with an active, known or suspected autoimmune disease which may affect vital organ function, or has/may require systemic immunosuppressive therapy for management are excluded. Participants with type I diabetes mellitus, hypothyroidism only requiring hormone replacement, skin disorders (such as vitiligo, psoriasis, or alopecia) not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger are permitted to enroll. * Participants with a condition requiring systemic treatment with either corticosteroids (\>10 mg daily prednisone equivalent) or other immunosuppressive medications within 7 days of start of study treatment. Inhaled or topical steroids, and adrenal replacement steroid doses \> 10 mg daily prednisone equivalent, are permitted in the absence of active autoimmune disease.