Cabozantinib S-Malate in Treating Younger Patients With Recurrent or Refractory Solid Tumors

Inclusion Criteria: * Patients must have a body surface area \>= 0.44 m\^2 when enrolling on dose level -1; patients must have a body surface area \>= 0.35 m\^2 when enrolling on dose level 1, 2, or 3 * PART A: Patients with relapsed or refractory solid tumors (excluding medullary thyroid cancer) including CNS tumors and malignant melanoma are eligible; patients must have had histologic verification of malignancy at original diagnosis or relapse except in patients with intrinsic brain stem tumors, optic pathway gliomas, or patients with pineal tumors and elevations of cerebrospinal fluid (CSF) or serum tumor markers including alpha-fetoprotein or beta-human chorionic gonadotropin (HCG) * Part B: Patients with medullary thyroid cancer (MTC), with or without bone marrow involvement, will be eligible for Part B; these patients will be enrolled at dose level 2, the recommended phase 2 dose determined in the dose escalation part of the study * Patients must have either measurable or evaluable disease * Patient's current disease state must be one for which there is no known curative therapy or therapy proven to prolong survival with an acceptable quality of life * Karnofsky \>= 50% for patients \> 16 years of age and Lansky \>= 50 for patients =\< 16 years of age * Note: neurologic deficits in patients with CNS tumors must have been relatively stable for at least 7 days prior to study enrollment; patients who are unable to walk because of paralysis, but who are up in a wheelchair, will be considered ambulatory for the purpose of assessing the performance score * Patients must have fully recovered from the acute toxic effects of all prior anti-cancer chemotherapy: * Myelosuppressive chemotherapy: at least 21 days after the last dose of myelosuppressive chemotherapy (42 days if prior nitrosourea) * Hematopoietic growth factors: at least 14 days after the last dose of a long-acting growth factor (e.g. Neulasta) or 7 days for short-acting growth factor; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair * Biologic (anti-neoplastic agent): at least 7 days after the last dose of a biologic agent; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair * Immunotherapy: at least 42 days after the completion of any type of immunotherapy, e.g. tumor vaccines * Monoclonal antibodies: at least 3 half-lives of the antibody after the last dose of a monoclonal antibody * Radiation therapy (XRT): at least 14 days after local palliative XRT (small port); at least 150 days must have elapsed if prior total-body irradiation (TBI), craniospinal XRT or if \>= 50% radiation of pelvis; at least 42 days must have elapsed if other substantial bone marrow (BM) radiation * Stem cell infusion without TBI: no evidence of active graft versus (vs.) host disease and at least 56 days must have elapsed after transplant or stem cell infusion * For patients with solid tumors without known bone marrow involvement: * Peripheral absolute neutrophil count (ANC) \>= 1000/mm\^3 * Platelet count \>= 100,000/mm\^3 (transfusion independent, defined as not receiving platelet transfusions for at least 7 days prior to enrollment) * Patients with known bone marrow metastatic disease will be eligible for study provided they meet the blood counts (may receive transfusions provided they are not known to be refractory to red cell or platelet transfusions); these patients will not be evaluable for hematologic toxicity; at least 5 of every cohort of 6 patients with a solid tumor must be evaluable for hematologic toxicity in the dose-escalation part of the study; if dose-limiting hematologic toxicity is observed, all subsequent patients enrolled must be evaluable for hematologic toxicity * Creatinine clearance or radioisotope glomerular filtration rate (GFR) \>= 70 ml/min/1.73 m\^2 or a serum creatinine based on age/gender as follows: * 2 to \< 6 years: 0.8 mg/dL * 6 to \< 10 years: 1 mg/dL * 10 to \< 13 years: 1.2 mg/dL * 13 to \< 16 years: 1.5 mg/dL (male), 1.4 mg/dL (female) * \>= 16 years: 1.7 mg/dL (male), 1.4 mg/dL (female) * Urine protein: =\< 30 mg/dl in urinalysis or =\< 1+ on dipstick, unless quantitative protein is \< 1000 mg in a 24 hour (h) urine sample * Bilirubin (sum of conjugated + unconjugated) =\< 1.5 x upper limit of normal (ULN) for age * Serum glutamate pyruvate transaminase (SGPT) (alanine aminotransferase \[ALT\]) =\< 110 U/L; for the purpose of this study, the ULN for SGPT is 45 U/L * Serum albumin \>= 2.8 g/dL * Prothrombin time (PT) and international normalized ratio (INR) =\< 1.5 x ULN * Serum amylase =\< 1.5 x ULN * Serum lipase =\< 1.5 x ULN * A blood pressure (BP) =\< the 95th percentile for age, height, and gender, and not receiving medication for treatment of hypertension; please note that 3 serial blood pressures should be obtained and averaged to determine baseline BP * Central nervous system function defined as: patients with seizure disorder may be enrolled if receiving non-enzyme inducing anticonvulsants and well controlled * No history of congenital prolonged corrected QT interval (QTc) syndrome, New York Heart Association (NYHA) class III or IV congestive heart failure (CHF) * No clinically significant cardiac arrhythmias, stroke or myocardial infarction within 6 months prior to enrollment * QTc =\< 480 msec; Note: patients with grade 1 prolonged QTc (450-480 msec) at the time of study enrollment should have correctable causes of prolonged QTc addressed if possible (i.e. electrolytes, medications) * All patients and/or their parents or legally authorized representatives must sign a written informed consent; assent, when appropriate, will be obtained according to institutional guidelines * Archival tumor tissue slides from either initial diagnosis or relapse must be sent; if tumor tissue is unavailable, the study chair must be notified prior to enrollment Exclusion Criteria: * Pregnant or breast-feeding women will not be entered on this study; pregnancy tests must be obtained in girls who are post-menarchal; males or females of reproductive potential may not participate unless they have agreed to use two methods of birth control - a medically accepted barrier method of contraceptive method (e.g., male or female condom) and a second effective contraceptive method of birth control - during protocol therapy and for at least 4 months after the last dose of XL184; abstinence is an acceptable method of birth control * Patients receiving corticosteroids who have not been on a stable or decreasing dose of corticosteroid for at least 7 days prior to enrollment are not eligible * Patients who are currently receiving another investigational drug are not eligible * Patients who are currently receiving other anti-cancer agents are not eligible * Patients who are receiving cyclosporine, tacrolimus or other agents to prevent graft-versus-host disease post bone marrow transplant are not eligible for this trial * Patients must not be receiving any of the following potent cytochrome P450 family 3, subfamily A, polypeptide 4 cytochrome (CYP3A4) inducers or inhibitors: erythromycin, clarithromycin, ketoconazole, azithromycin, itraconazole, grapefruit juice or St. John's wort * Patients who are receiving systemic treatment anticoagulation are not eligible; patients receiving prophylactic systemic anticoagulation will be allowed as long as eligibility PT/INR requirements are met * Patients must not have received enzyme-inducing anticonvulsants within 14 days prior to enrollment * Patients who are receiving drugs that prolong QTc are not eligible * Patients must be able to swallow intact tablets; patients who cannot swallow intact tablets are not eligible * Patients with active bleeding are not eligible; specifically, no clinically significant gastrointestinal (GI) bleeding, GI perforation, intra-abdominal abscess or fistula for 6 months prior to enrollment, no hemoptysis or other signs of pulmonary hemorrhage for 3 months prior to enrollment * Patients with evidence of an acute intracranial or intratumoral hemorrhage on computed tomography (CT) or magnetic resonance imaging (MRI) are not eligible (patients with evidence of resolving hemorrhage will be eligible) * Patients who have had or are planning to have the following invasive procedures are not eligible: * Major surgical procedure, laparoscopic procedure, open biopsy or significant traumatic injury within 28 days prior to enrollment * Central line placement or subcutaneous port placement is not considered major surgery but must be placed at least 3 days prior to enrollment for external lines (e.g. Hickman or Broviac) and at least 7 days prior to enrollment for subcutaneous port * Core biopsy within 7 days prior to enrollment * Fine needle aspirate within 7 days prior to enrollment * Surgical or other wounds must be adequately healed prior to enrollment * Patients on antihypertensive therapy for control of blood pressure at the time of enrollment are not eligible * Patients with any medical or surgical conditions that would interfere with gastrointestinal absorption of this oral agent are not eligible * Patients who have an uncontrolled infection are not eligible * Patients who have received a prior solid organ transplantation are not eligible * Patients who in the opinion of the investigator may not be able to comply with the safety monitoring requirements of the study are not eligible