Wild-Type Reovirus in Combination With Sargramostim in Treating Younger Patients With High-Grade Relapsed or Refractory Brain Tumors

Inclusion Criteria: * Histological confirmation of a high grade (grade 3 or 4) primary brain tumor either classified as a glioma (including astrocytoma, anaplastic oligodendroglioma and glioblastoma multiforme), medulloblastoma, atypical teratoid/rhabdoid tumor (AT/RT) or primitive neuroectodermal tumor (PNET) * Note: Patients with diffuse intrinsic pontine glioma (DIPG) are exempt from this confirmation of tumor if characteristic radiologic findings are noted on magnetic resonance imaging (MRI) * Patients must have no known curative therapy available * Evidence of tumor progression by MRI scan following radiation therapy or following the most recent anti-tumor therapy; note: patients who have had surgical treatment at recurrence are eligible if they had a resection with measurable residual disease on postoperative imaging or if there is imaging evidence of disease progression as compared to the first postoperative scan * Measurable disease: measurable by gadolinium MRI scan * Absolute neutrophil count (ANC) \>= 750 /uL obtained =\< 7 days prior to registration * Absolute lymphocyte count (ALC) \>= 250/uL obtained =\< 7 days prior to registration * Platelet count (PLT) \>= 75,000 /uL without transfusions obtained =\< 7 days prior to registration * Hemoglobin \>= 7.0 gm/dL obtained =\< 7 days prior to registration * Total bilirubin =\< 1.5 times upper limit of institutional normal (ULN) for age obtained =\< 7 days prior to registration * Aspartate transaminase (AST) =\< 3 times ULN for age obtained =\< 7 days prior to registration * Serum albumin \>= 2 g/dL * Creatinine =\< 1.5 times ULN for age OR a creatinine clearance or glomerular filtration rate (GFR) \>= 70 mL/min/1.73 m\^2; obtained =\< 7 days prior to registration * Karnofsky or Lansky performance status (PS): performance status of \>= 50 assessed within two weeks prior to registration; neurological deficits in patients must have been relatively stable for a minimum of 1 week prior to study enrollment; patients who are unable to walk because of paralysis, but who are up in a wheelchair, will be considered ambulatory for the purpose of assessing the performance score * Ability to understand and the willingness to provide written informed assent or consent * Willing to return to enrolling institution for follow-up * Immunosuppressants: patients must be receiving a stable or decreasing dose of dexamethasone for at least 1 week prior to start of therapy AND dexamethasone dose must be =\< 0.1 mg/kg/day AND =\< a total daily dose of 4 mg/day * If patient has a clinically indicated surgery or biopsy at any time during treatment with Reolysin, a tissue sample will be collected for correlative research purposes * Patient willing to provide mandatory blood samples for correlative research purposes; the inability to provide a blood sample or the lack of an available blood sample does not make the patient ineligible * Negative urine or serum pregnancy test done =\< 7 days prior to registration for females who are post-menarchal * Patient agrees to use an acceptable form of contraception during the study and for up to 28 days after the last dose of Reolysin if patient or female partner is post-menarche; acceptable methods include 1) a double barrier method, such as condom and spermicide; 2) hormonal contraception methods, including pills, patches, rings, or injections except progestin-only containing pills (i.e., 'mini-pill'); 3) intrauterine device (non-progesterone T); 4) surgical methods such as a bilateral tubal ligation or a vasectomy; 5) abstinence * Must be able to avoid direct contact with pregnant women, infants \< 3 months of age and immunocompromised individuals while on study and for \>= 3 weeks following the last dose of study agent administration; direct contact is defined as household contact, i.e., anyone living with the patient * Life expectancy \>= 3 months Exclusion Criteria: * Fetal and newborn toxicity: any of the following * Pregnant women * Nursing women * Males or post-menarchal females who are unwilling to employ adequate contraception throughout the duration of the study and for at least 4 weeks after treatment has ended * Uncontrolled intercurrent illness including, but not limited to: * Ongoing or active infection, including localized infections * Symptomatic congestive heart failure * Unstable angina pectoris or cardiac arrhythmia * Any psychiatric illness/social situations that would limit compliance with study requirements * Co-morbid systemic illnesses or other severe concurrent disease which, in the judgment of the investigator, would make the patient inappropriate for entry into this study or interfere significantly with the proper assessment of safety and toxicity of the prescribed regimens * Known prior history of tuberculosis or positive purified protein derivative (PPD) test result * Known prior history of human immunodeficiency virus (HIV) * Administration of live vaccines =\< 14 days prior to registration; note: patients may not receive any viral immunizations during the study and for 28 days after the last dose of Reolysin * Prior history of any viral-based therapy * Other concurrent chemotherapy, immunotherapy, radiotherapy, any ancillary therapy considered investigational (utilized for a non-Food and Drug Administration \[FDA\]-approved indication and in the context of a research investigation) or receiving any other investigational agent which would be considered as a treatment for the primary neoplasm: * Chemotherapy =\< 3 weeks of registration * Nitrosoureas or mitomycin C =\< 6 weeks of registration * Small molecule cell cycle inhibitors =\< 2 weeks prior to registration * Immunotherapy =\< 6 weeks prior to registration * Monoclonal antibodies =\< 3 half-lives prior to registration * Radiation therapy * Last fraction of craniospinal irradiation or total body irradiation =\< 3 months prior to registration or last fraction of focal irradiation to symptomatic metastatic sites =\< 4 weeks prior to registration * Growth factors * Colony forming growth factors \< 2 weeks prior to registration (i.e., filgrastim, sargramostim, erythropoietin) * Neulasta \< 2 weeks prior to registration * Failure to fully recover from acute, reversible effects of prior chemotherapy regardless of interval since last treatment