Testing the Addition of Radiation Therapy to Immunotherapy for Merkel Cell Carcinoma

Inclusion Criteria: * Patients must have pathologically (histologically or cytologically) proven diagnosis of MCC by local pathology review * Have measurable disease based on RECIST 1.1 including at least two cancerous deposits; at least one deposit must be RECIST measurable while at least one deposit must meet criteria for SBRT; non-radiated tumor will be identified prior to randomization on the protocol * Patients must have advanced or metastatic MCC defined as evidence of distant metastasis(es) on imaging * Patients with locoregionally confined disease are not eligible * No prior immunotherapy for advanced/metastatic MCC * Patients with known or suspected central nervous system (CNS) metastases, untreated CNS metastases, or with the CNS as the only site of disease are excluded; however, subjects with controlled brain metastases will be allowed to enroll; controlled brain metastases are defined as no radiographic progression for at least 4 weeks following radiation and/or surgical treatment (or 4 weeks of observation if no intervention is clinically indicated), and off of steroids for at least 2 weeks, and no new or progressive neurological signs and symptoms * Patients having received palliative radiotherapy for extracranial metastasis(es) are eligible as long as there are 2 cancerous deposits that have not received prior radiation therapy (RT) and they meet the following criteria * No prior radiation therapy (\> 5 Gy) to the metastasis intended to be treated with SBRT * No history of the following: * Autoimmunity requiring systemic immunosuppression within 2 years * Patients known to be human immunodeficiency virus (HIV) positive are eligible if they meet the following: * CD4 counts \>= 350 mm\^3 * Serum HIV viral load of \< 25,000 IU/ml * No other active malignancy that the investigator determines would interfere with the treatment and safety analysis * Not pregnant and not nursing, because this study involves an investigational agent whose genotoxic, mutagenic and teratogenic effects on the developing fetus and newborn are unknown; therefore, for women of childbearing potential only, a negative (if your test schedule specifically indicates a urine or serum pregnancy test, add that information at this point) pregnancy test done =\< 28 days prior to registration is required * Eastern Cooperative Oncology Group (ECOG) performance status 0-2 * Absolute neutrophil count (ANC) \>= 1,500/mm\^3 * Platelet count \>= 100,000/mm\^3 * Hemoglobin \>= 9.0 g/dl * Total bilirubin =\< 2.0 mg/dl * Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 3.0 x upper limit of normal (ULN) * Systolic blood pressure (BP) =\< 150 mg HG * Diastolic BP =\< 90 mg HG * Albumin \> 3 mg/dl * Blood urea nitrogen (BUN) =\< 30 mg/dl * Creatinine =\< 1.7 mg/dl * The following imaging workup to document metastases within 45 days prior to study registration are required: CT scans of the chest, abdomen and pelvis with radionuclide bone scan OR whole body (at least skull base to midthigh) positron emission tomography (PET)/CT