Temsirolimus, Irinotecan Hydrochloride, and Temozolomide in Treating Younger Patients With Relapsed or Refractory Solid Tumors

Inclusion Criteria: * Patients must have had histologic verification of malignancy at original diagnosis or relapse except in patients with intrinsic brain stem tumors, patients with optic pathway gliomas, and patients with pineal tumors and elevations of serum or cerebrospinal fluid (CSF) alpha-fetoprotein or beta-human chorionic gonadotropin (HCG) * Patients must have either measurable or evaluable disease * Patient's current disease state must be one for which there is no known curative therapy or therapy proven to prolong survival with an acceptable quality of life * Karnofsky \>= 50% for patients \> 16 years of age and Lansky \>= 50 for patients =\< 16 years of age; Note: neurologic deficits in patients with central nervous system (CNS) tumors must have been relatively stable for a minimum of 1 week prior to study enrollment; patients who are unable to walk because of paralysis, but who are up in a wheelchair, will be considered ambulatory for the purpose of assessing the performance score * Patients must have fully recovered from the acute toxic effects of all prior anti-cancer chemotherapy; * Myelosuppressive chemotherapy: patients must not have received myelosuppressive therapy within 3 weeks of enrollment onto this study (6 weeks if prior nitrosourea) * Hematopoietic growth factors: at least 14 days after the last dose of a long-acting growth factor (e.g. Neulasta) or 7 days for short-acting growth factor; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair * Biologic (anti-neoplastic agent): at least 7 days after the last of a biologic agent that is not a monoclonal antibody and enrollment on this study; for agents that have known adverse events occurring beyond 7 days after administration, this period must be extended beyond the time during which adverse events are known to occur; the duration of this interval must be discussed with the study chair * Immunotherapy: at least 6 weeks since the completion of any type of immunotherapy, e.g. tumor vaccines * Monoclonal antibodies: at least 3 half-lives must have elapsed after treatment with a monoclonal antibody and enrollment on this study * Radiation therapy (XRT): \>= 2 weeks must have elapsed for local palliative XRT (small port) and enrollment on study; at least 24 weeks must have elapsed since radiation if prior total body irradiation (TBI), craniospinal XRT or if radiation to \>= 50% radiation of pelvis has been administered; \>= 6 weeks must have elapsed if the patient has received other substantial bone marrow (BM) radiation * Stem Cell Infusion without TBI: the patient must have no evidence of active graft versus (vs.) host disease, and \>= 12 weeks must have elapsed since transplant or stem cell infusion and enrollment on this study * Prior treatment with irinotecan, temozolomide, or temsirolimus: patients previously treated with any of these drugs as single agents will be eligible for this study; patients previously treated with two of the three drugs (including irinotecan + temozolomide) will also be eligible, however patients previously treated with all three agents in combination will not be eligible * Peripheral absolute neutrophil count (ANC) \>= 1,000/mm\^3 * Platelet count \>= 100,000/mm\^3 (transfusion independent defined as not receiving platelet transfusions within a 7 day period prior to enrollment) * Creatinine clearance or radioisotope glomerular filtration rate (GFR) \>= 70 mL/min OR a serum creatinine based on age and/or gender as follows: * 0.6 mg/dL (1 to \< 2 years of age) * 0.8 mg/dL (2 to \< 6 years of age) * 1.0 mg/dL (6 to \< 10 years of age) * 1.2 mg/dL (10 to \< 13 years of age) * 1.5 mg/dL (male) or 1.4 mg/dL (female) (13 to \< 16 years of age) * 1.7 mg/dL (male) or 1.4 mg/dL (female) (\>= 16 years of age) * Bilirubin (sum of conjugated + unconjugated) =\< 1.5 times upper limit of normal (ULN) * Serum glutamate pyruvate transaminase (SGPT) (alanine aminotransferase \[ALT\]) =\< 110 U/L; for the purpose of this study, the ULN for SGPT is 45 U/L * Serum albumin \> 2 g/dL * Prothrombin time (PT) \< 1.2 times ULN * Serum triglyceride level =\< 300 mg/dL * Serum cholesterol =\< 300 mg/dL * Random or fasting blood glucose within the upper normal limits for age; if the initial blood glucose is a random sample that is outside of the normal limits, then a follow-up fasting blood glucose can be obtained and must be within the upper normal limits for age * Normal pulmonary function tests, including diffusion capacity of carbon monoxide (DLCO), if there is clinical indication for determination (e.g., dyspnea at rest, known requirement for supplemental oxygen); for patients who do not have respiratory symptoms, full pulmonary function tests (PFTs) are NOT required * Patients with seizure disorder may be enrolled if on non-enzyme inducing anticonvulsants and if seizures are well controlled * Nervous system disorders (Common Terminology Criteria for Adverse Events \[CTCAE\] version 4.0 \[v4\]) resulting from prior therapy must be =\< grade 2 * All patients and/or their parents or legal guardians must sign a written informed consent; assent, when appropriate, will be obtained according to institutional guidelines Exclusion Criteria: * Pregnant or breast-feeding women will not be entered on this study; pregnancy tests must be obtained in girls who are post-menarchal; males or females of reproductive potential may not participate unless they have agreed to use an effective contraceptive method * Patients receiving chronic systemic corticosteroids are not eligible; patients must have been off systemic corticosteroids for 7 days prior to enrollment * Patients who are currently receiving another investigational drug are not eligible * Patients who are currently receiving other anti-cancer agents are not eligible * Patients who are currently receiving enzyme inducing anticonvulsants are not eligible * Patients must not be receiving any of the following potent Cytochrome P450 family 3, subfamily A, polypeptide 4 (CYP3A4) inducers or inhibitors: erythromycin, clarithromycin, ketoconazole, azithromycin, itraconazole, grapefruit juice or St. John's worth * Patients who are currently receiving therapeutic anticoagulants (including aspirin, low molecular weight heparin, and others) are not eligible * Patients who are currently receiving angiotensin-converting enzyme (ACE) inhibitors are not eligible * Patients who are receiving cyclosporine, tacrolimus or other agents to prevent either graft-versus-host disease post bone marrow transplant or organ rejection post-transplant are not eligible for this trial. * Patients who have an uncontrolled infection are not eligible * Patients who in the opinion of the investigator may not be able to comply with the safety monitoring requirements of the study are not eligible * Patients with history of allergic reactions attributed to compounds of similar composition to irinotecan hydrochloride, temozolomide, or temsirolimus are not eligible * Patients must not have had major surgery for 6 weeks prior to enrollment on study; patients with history of recent minor surgical procedures (vascular catheter placement, bone marrow evaluation, laparoscopic surgery and the like) will be eligible