Testing the Combination of XL184 (Cabozantinib), Nivolumab, and Ipilimumab for Poorly Differentiated Neuroendocrine Tumors

Inclusion Criteria: * Patients must have metastatic, histologically confirmed poorly-differentiated neuroendocrine neoplasms per 2018 World Health Organization (WHO) classification, with the exception of small cell lung cancer and merkel cell carcinoma. All variations of poorly differentiated neuroendocrine carcinoma (small cell, large cell and mixed cells) are eligible * Failure of only one line of prior systemic cancer treatment * Patients must have measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1 * Patients must have lesions that can be safely biopsied and be willing to have a pre-treatment and an on-treatment biopsy (after 1 month of treatment with the combination regimen) and a blood collection at baseline * Prior systemic cancer therapy must have been completed at least 4 weeks prior to cycle 1 day 1 of treatment with the combination regimen * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 (Karnofsky \>= 60%) * Leukocytes \>= 3,000/mcL * Absolute neutrophil count \>= 1,500/mcL without granulocyte-colony stimulating (GCSF) factor support * Hemoglobin \>= 9 g/dL * Serum thyroid stimulating hormone (TSH) within institutional normal limits * Platelets \>= 100,000/mcL * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN); =\< 3.0 x ULN for patients with Gilbert's syndrome * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3.0 x institutional ULN * Alkaline phosphatase =\< 3.0 x institutional ULN; =\< 5.0 x ULN with documented bone metastases * Creatinine =\< institutional ULN OR creatinine clearance (CrCl) \>= 50 mL/min/1.73 m\^2 (using the Cockcroft-Gault formula) * Serum albumin \>= 2.8 g/dL * Lipase \< 2.0 x ULN and no radiologic or clinical evidence of pancreatitis * Urine protein/creatinine ratio (UPCR) =\< 1 mg protein/mg creatinine * Serum phosphorus, calcium, magnesium, and potassium within institutional normal limits * Prothrombin time (PT)/international normalized ratio (INR) and partial thromboplastin time (PTT) test \< 1.3 x ULN * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy utilizing agents that do not strongly induce or inhibit cytochrome P450 (CYP) 3A4 with undetectable viral load within 6 months prior to study registration are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression and are off steroid support for at least 4 weeks after treatment for metastases is complete and within 28 days prior to the first dose of study treatment. Radiotherapy should have been stopped at least 4 weeks prior to study registration. Brain surgery should not have occurred within 3 months of study registration to be eligible * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * The effects of XL184 (cabozantinib), nivolumab, and ipilimumab on the developing human fetus are unknown. For this reason, women of child-bearing potential (WOCBP) and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. WOCBP should use an adequate method to avoid pregnancy for 5 months after the last dose of study therapy. Women of childbearing potential must have a negative serum or urine pregnancy test (minimum sensitivity: 25 IU/L or equivalent units of human chorionic gonadotropin \[hCG\]) within 24 hours prior to the start of study therapy. Women must not be breastfeeding. Men who are sexually active with WOCBP must use any contraceptive method with a failure rate of \< 1% per year. Men who receive study therapy and who are sexually active with WOCBP will be instructed to adhere to contraception for a period of 7 months after the last dose of study therapy. Women who are not of childbearing potential (i.e., who are postmenopausal or surgically sterile) as well as azoospermic men do not require contraception. WOCBP is defined as any female who has experienced menarche and who has not undergone surgical sterilization (hysterectomy or bilateral oophorectomy) or who is not postmenopausal. Menopause is defined clinically as 12 months of amenorrhea in a woman over 45 in the absence of other biological or physiological causes. In addition, women under the age of 55 must have a documented serum follicle stimulating hormone (FSH) level \< 40 mIU/mL. WOCBP and men who are sexually active with WOCBP will be instructed to adhere to contraception for a period of 5 and 7 months, respectively, after the last dose of study therapy. These durations have been calculated using the upper limit of the half-life for nivolumab (25 days) and are based on the protocol requirement that WOCBP use contraception for 5 half-lives plus 30 days and men who are sexually active with WOCBP use contraception for 5 half-lives plus 90 days. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she (or the participating partner) must inform the treating physician immediately * Patients must be able to swallow tablets * Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity (IDMC) who have a legally-authorized representative (LAR) and/or family member available will also be eligible Exclusion Criteria: * Patients must not require systemic corticosteroids treatment (\>= 10 mg/day prednisone equivalents) or other immunosuppressive medications within 28 days prior to study drug administration. Inhaled or topical steroids and adrenal replacement doses \< 10 mg/day prednisone equivalents are permitted in the absence of active autoimmune disease. Patients are permitted to use topical, ocular, intra-articular, intranasal, and inhalational corticosteroids (with minimal systemic absorption). Physiologic replacement doses of systemic corticosteroids in patients with adrenal insufficiency are permitted, even if \>= 10 mg/day prednisone equivalents. A brief course of corticosteroids for prophylaxis or for treatment of non-autoimmune conditions (e.g., delayed-type hypersensitivity reaction caused by contact allergen) is permitted, as is steroid pre-medication for contrast allergy * Patients must not have had prior treatment with an anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CTLA-4 antibody, or any other antibody or drug specifically targeting T cell co-stimulation or immune checkpoint pathways * Patients must not have had prior treatment with XL184 (cabozantinib), or any MET-targeting tyrosine kinase inhibitor (TKI), or any MET-targeting monoclonal antibody (MetMAb), such as onartuzumab * Patients must not have received radiation therapy to any part of the body within 28 days * Patients must not have clinically relevant, ongoing complications from prior radiation therapy. No radiation therapy is allowed while the patient is on study. Palliative radiation therapy, if needed, should be completed at least 28 days prior to enrollment into the study as described above * Patients must not require concomitant treatment with oral anticoagulants (e.g., warfarin, direct thrombin, and factor Xa inhibitors) or platelet inhibitors (e.g., clopidogrel). The following anticoagulants are allowed: * Low-dose aspirin for cardioprotection (per local applicable guidelines), * Low-dose low molecular weight heparins (LMWH), * Therapeutic doses of LMWH are allowed in patients without known brain metastases who are on a stable dose of LMWH for at least 6 weeks before the first dose of study treatment, and who have had no clinically significant hemorrhagic complications from the anticoagulation regimen or the tumor * Patients must not have had major surgery (e.g., gastrointestinal \[GI\] surgery or removal or biopsy of brain metastasis) within 4 weeks before first dose of study treatment. Complete wound healing from major surgery must have occurred 1 month before the first dose of study treatment and from minor surgery (e.g., simple excision or tooth extraction) at least 10 days before the first dose. Patients with clinically relevant ongoing complications from prior surgery are not eligible * Patients must not have received cytotoxic chemotherapy (including investigational cytotoxic chemotherapy) or biologic agents (e.g., cytokines or antibodies) within 4 weeks, or nitrosoureas/ mitomycin C within 4 weeks, before the first dose of study treatment. Patients may continue on bone-modifying agents (denosumab or bisphosphonates) with caution * Patients who have not recovered to baseline from adverse events due to prior anti-cancer therapy (i.e., have residual toxicities \> grade 1) according to Common Terminology Criteria for Adverse Events (CTCAE) version (v)0.5 unless the adverse events (AEs) are clinically nonsignificant and/or stable on supportive therapy, with the exception of alopecia * Patients who are receiving any other investigational agents. Patients must not have received any other type of investigational agent within 4 weeks before the first dose of study treatment to be eligible * Patients must not have a corrected QT interval calculated by the Fridericia formula (QTcF) \> 500 msec by electrocardiogram (EKG) within 28 days before the first dose of study treatment * Note: If a single EKG shows a QTcF with an absolute value \> 500 msec, two additional EKGs at intervals of approximately 3 min must be performed within 30 min after the initial EKG, and the average of these three consecutive results for QTcF will be used to determine eligibility * Patients should not have known, untreated brain metastases or leptomeningeal metastases because of poor prognosis and concerns that progressive neurologic dysfunction could confound the evaluation of neurologic and other adverse events * History of allergic reactions attributed to compounds of similar chemical or biologic composition to agents used in study * Patients must not have a history of severe hypersensitivity reactions to any monoclonal antibodies * Patients must not require concomitant treatment with strong CYP3A4 inducers (e.g., dexamethasone, phenytoin, carbamazepine, rifampin, rifabutin, rifapentine, phenobarbital, or St. John's wort). Because lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference. As part of the enrollment/informed consent procedures, patients will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Patients must not have uncontrolled, significant intercurrent or recent illness including, but not limited to, the following conditions: * Cardiovascular disorders: * Congestive heart failure New York Heart Association (NYHA) Class 3 or 4, unstable angina pectoris, serious cardiac arrhythmias. * Uncontrolled hypertension defined as sustained blood pressure (BP) \> 140 mm Hg systolic or \> 90 mm Hg diastolic despite optimal antihypertensive treatment within seven days prior to the first dose of study treatment. * Stroke (including transient ischemic attack \[TIA\]), myocardial infarction (MI), or other ischemic event, or thromboembolic event (e.g., deep venous thrombosis \[DVT\], pulmonary embolism \[PE\]) within 6 months before first dose. * GI disorders including those associated with a high risk of perforation or fistula formation: * The patient has evidence of tumor invading the GI tract, active peptic ulcer disease, inflammatory bowel disease (e.g., Crohn's disease), diverticulitis, cholecystitis, symptomatic cholangitis or appendicitis, acute pancreatitis, acute obstruction of the pancreatic duct or common bile duct, or gastric outlet obstruction. * Abdominal fistula, GI perforation, bowel obstruction, or intra-abdominal abscess within 6 months before first dose. Complete healing of an intra-abdominal abscess must be confirmed before first dose. * Clinically significant hematuria, hematemesis, or hemoptysis or other history of significant bleeding (e.g., pulmonary hemorrhage) within 12 weeks before first dose. * Cavitating pulmonary lesion(s) or known endotracheal or endobronchial disease manifestation. * Lesions invading or encasing any major blood vessels. * Other clinically significant disorders that would preclude safe study participation. * Serious non-healing wound/ulcer/bone fracture. * Uncompensated/symptomatic hypothyroidism. * Moderate to severe hepatic impairment (Child-Pugh B or C). * Patients with psychiatric illness/social situations that would limit compliance with study requirements * Pregnant women are excluded from this study because XL184 (cabozantinib) has the potential for teratogenic or abortifacient effects, and the effects of nivolumab and ipilimumab on the developing fetus are not well known. Because there is an unknown but potential risk for AEs in nursing infants secondary to treatment of the mother, breastfeeding must be discontinued if the mother is treated with XL184 (cabozantinib), nivolumab, or ipilimumab * Patients with active autoimmune disease or history of autoimmune disease that might recur, which may affect vital organ function or require immune suppressive treatment including high dose systemic corticosteroids, should be excluded. These include but are not limited to: immune-related neurologic disease, such as multiple sclerosis, autoimmune (demyelinating) neuropathy, Guillain-Barre syndrome, or myasthenia gravis; systemic autoimmune disease such as systemic lupus erythematosus (SLE), connective tissue diseases, scleroderma, inflammatory bowel disease (IBD), Crohn's, ulcerative colitis, or autoimmune hepatitis. Patients with a history of toxic epidermal necrolysis (TEN), Stevens-Johnson syndrome, or phospholipid syndrome should be excluded because of the risk of recurrence or exacerbation of disease. Patients with vitiligo, type I diabetes mellitus (DM), or endocrine deficiencies (e.g., thyroiditis) managed with replacement hormones, including physiologic corticosteroids, are eligible. Patients with rheumatoid arthritis and other arthropathies, Sjogren's syndrome and psoriasis controlled with topical medication, and patients with positive serology, (e.g., antinuclear antibodies \[ANA\] or anti-thyroid antibodies) should be evaluated for the presence of target organ involvement and potential need for systemic treatment but should otherwise be eligible