Vorinostat and Temsirolimus With or Without Radiation Therapy in Treating Younger Patients With Newly Diagnosed or Progressive Diffuse Intrinsic Pontine Glioma

Inclusion Criteria: * Patients must be \> than 6 months and =\< 21 years of age at the time of study consent * Patients with newly diagnosed or progressive DIPG as confirmed by gadolinium enhanced magnetic resonance imaging (MRI) are eligible; MRI must demonstrate that at least 2/3 of the tumor is situated in the pons and that the origin of the tumor is clearly in the pons; biopsy is not required; tumors with features not typical of diffuse intrinsic brainstem glioma are not eligible; these include dorsally exophytic brainstem gliomas, cervicomedullary junction tumors, and focal low grade gliomas of the midbrain or brainstem which should undergo resection and pathologic evaluation; patients, who have received re-irradiation for progression of the tumor, will be eligible if they show evidence of measurable progressive disease after the re-irradiation; patients at diagnosis with involvement of the spine will not be eligible, however if at progression features of spine involvement are present they will be eligible for stratum II * Patients must have a Karnofsky \>= 50% for patients \> 16 years of age and Lansky \>= 50 for patients=\< 16 years of age; patients who are unable to walk because of paralysis, but who are up in a wheelchair, will be considered ambulatory for the purpose of assessing the performance score * Patients must have fully recovered from the acute toxic effects of all prior anti-cancer chemotherapy or radiation to grade 2 or less * Patients must not have received myelosuppressive therapy within 3 weeks of enrollment onto this study (6 weeks if prior nitrosourea) * At least 14 days must have passed after the last dose of a long-acting growth factor (e.g. Neulasta) or 7 days for short-acting growth factor * At least 7 days must have elapsed after the last of a biologic agent that is not a monoclonal antibody, to be enrolled on this study * At least 6 weeks since the completion of any type of immunotherapy, e.g. tumor vaccines * At least 3 half-lives must have elapsed after treatment with a monoclonal antibody and enrollment on this study * \>= 2 weeks must have elapsed for local palliative radiotherapy (re-irradiation for progressive disease or upfront radiation therapy \[RT\] at initial diagnosis) and enrollment on study for stratum II; at least 24 weeks must have elapsed if patient received craniospinal radiotherapy due to any other prior malignancies * The patient must have no evidence of active graft vs. host disease, and \>= 12 weeks must have elapsed since transplant or stem cell infusion and enrollment on this study for any other pathology * Prior treatment with vorinostat is allowed but at least 3 weeks must have elapsed from the last dose and effects of prior therapy have resolved * Patients with central nervous system (CNS) tumors who are receiving steroids are eligible * Peripheral absolute neutrophil count (ANC) \>= 1000/uL * Platelet count \>= 100,000/uL (transfusion independent) * Hemoglobin \>= 10.0 gm/dL (transfusion independent) * Serum creatinine =\< 1.5 x institutional upper limit of normal for age * Bilirubin (sum of conjugated + unconjugated) less than 1.5 x upper limit of normal (ULN) for age * Serum glutamate pyruvate transaminase (SGPT) (alanine aminotransferase \[ALT\]) =\< 110 U/L; for the purpose of this study, the ULN for SGPT is 45 U/L * Serum albumin must be \>=2 g/dL * Prothrombin time (PT) and international normalized ratio (INR) \< 1.2 x ULN * Patients with seizure disorder may be enrolled if on non-enzyme inducing anticonvulsants and well controlled * Serum cholesterol and serum triglyceride levels must be less than 300 mg/dl * Females \> 13 years of age or who have achieved menarche must have a negative pregnancy test within 2 weeks of starting treatment (urine or serum) to be eligible and if sexually active must also agree to use contraception; male sexually active patients must agree to use an effective method of contraception * Patient's current disease state must be one for which there is no known curative therapy or therapy proven to prolong survival with an acceptable quality of life * Patients must have a life expectancy of \>= 2 months; neurologic deficits in patients with CNS tumors must have been relatively stable for a minimum of 1 week prior to starting protocol therapy Exclusion Criteria: * Patients with other malignancies will not be eligible for stratum I or II; patients with disseminated disease including to the spine will not be eligible for stratum 1 but will be eligible for stratum II * Patients must not have a history of myocardial infarction, severe or unstable angina, clinically significant peripheral vascular disease, grade 2 or greater heart failure, or serious and inadequately controlled cardiac arrhythmia * Pregnant or breast-feeding women will not be entered on this study due to risks of fetal and teratogenic adverse events as seen in animal/human studies * Patients who are currently receiving enzyme inducing anticonvulsants are not eligible * Patients who are currently receiving therapeutic anticoagulants (including aspirin, low molecular weight heparin, and others) are not eligible * Patients who are currently receiving angiotensin-converting enzyme (ACE) inhibitors are not eligible due to the development of angioneurotic edema-type reactions in some subjects who received concurrent treatment with temsirolimus + ACE inhibitors * Patients who are receiving cyclosporine, tacrolimus or other agents to prevent either graft-versus-host disease post bone marrow transplant or organ rejection post-transplant are not eligible for this trial * Patients with history of allergic reactions attributed to compounds of similar chemical; or biologic composition to vorinostat or temsirolimus are not eligible * Patients who have an uncontrolled infection are not eligible * Patients who in the opinion of the investigator may not be able to comply with the safety monitoring requirements of the study are not eligible * Patients with newly diagnosed DIPG who have received vorinostat previously will not be eligible for stratum I; patients with progressive DIPG will be eligible if they have received either one of the two drugs vorinostat or temsirolimus but will not be eligible for stratum II if have received both the drugs before