SRS and Nivolumab in Treating Patients With Newly Diagnosed Melanoma Metastases in the Brain or Spine

Inclusion Criteria: * Patients must have histologically confirmed diagnosis of melanoma; the pathologic confirmation may be from another metastatic site or from metastatic brain or spine lesions * Patients must have stage IV melanoma, with newly identified brain or spine metastases * Patients must have measurable lesion in the brain or spine that is \>= 3 mm seen on magnetic resonance imaging (MRI) with contrast; NOTE: contrasted pre-treatment MRI scan must be obtained =\< 21 days prior to stereotactic radiosurgery treatment * Karnofsky performance scale \>= 70% * Leukocytes \>= 3,000/mcL * Absolute neutrophil count \>= 1,500/mcL * Platelets \>= 100,000/mcL * Total bilirubin =\< 2 x institutional upper limit of normal * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/ alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal * Creatinine within normal institutional limits OR according to Johns Hopkins MRI policy * Women of child bearing potential (WOCBP) must use a reliable form of contraception during the study treatment period and for up to 12 weeks following the last dose of study drug * Men must agree to the use of male contraception during the study treatment period and for at least 12 weeks after the last dose of study drug * Ability to understand and the willingness to sign written informed consent document(s) Exclusion Criteria: * Prior whole brain radiation or conventional radiation to the spine at the site of new lesion * Prior chemotherapy within 28 days of starting treatment * Prior therapy with investigational drugs within 28 days or at least 5 half-lives (whichever is longer) before study administration * Prior therapy with an anti- programmed cell death 1 (PD-1), anti- programmed cell death-ligand 1 (PD-L1), or anti-PDL-2 antibody * Neurologic dysfunction that would confound the evaluation of neurologic and other adverse events * Known allergy to compounds of similar chemical or biologic composition to nivolumab * Pregnant or breastfeeding women * Known history of human immunodeficiency virus * Active infection requiring therapy, positive tests for hepatitis B surface antigen or hepatitis C ribonucleic acid (RNA) * Active autoimmune disease, history of autoimmune disease or history of syndrome that required systemic steroids or immunosuppressive medications, e.g. organ, tissue, or allogenic hematopoietic stem cell transplant (HSCT) recipients. Exceptions include those with vitiligo or resolved childhood asthma/atopy. Subjects with asthma who require intermittent use of bronchodilators (such as albuterol) will not be excluded from this study * Use of any live vaccines against infectious diseases up to 4 weeks (28 days) before receiving nivolumab. (NOTE: Inactivated seasonal influenza vaccines are permitted and do not require a 4-week waiting period before starting study treatment). * Prisoners or subjects who are compulsorily detained for treatment of either a psychiatric or physical (e.g. infectious disease) illness * Patients with both brain and spine metastases will be excluded from the trial