Study of Avelumab and/or Radiation Therapy in People With Advanced Merkel Cell Carcinoma

Inclusion Criteria: * Biopsy proven Merkel cell carcinoma which is unresectable or metastatic, stage III or IV * Prior first-line treatment with aPD1 monotherapy (defined as at least one dose of pembrolizumab, avelumab, nivolumab, etc.) with evidence of progression of disease ≥10 weeks after starting therapy, in the absence of significant clinical deterioration * Patients with clinical deterioration during aPD1 monotherapy are eligible ≥6 weeks after starting aPD1 therapy * Criteria for clinical deterioration to be determined and agreed upon by treating physician and Principal Investigator * All detectable sites of MCC are amenable to comprehensive ablative radiation therapy in opinion of treating radiation oncologist and principal investigator * ≥18 years of age * Performance status ≤2 on the Eastern Cooperative Oncology Group Performance Scale * Able to provide valid written informed consent * Normal organ and marrow function * Hematologic: Neutrophil count ≥1500/mm\^3, platelet count ≥100,000/mm\^3, hemoglobin ≥9 g/dL * Hepatic: Total bilirubin ≤ 1.5 times the upper limit of normal, unless Gilbert's syndrome; aspartate transaminase and alanine transaminase ≤ 2.5 times the upper limit of normal (in the absence of hepatobiliary metastases); ≤ 3.0 times the upper limit of normal (in the presence of hepatobiliary metastases) * Renal: Estimated creatinine clearance ≥ 30 mL/min according to the Cockcroft-Gault formula. Exclusion Criteria: * Prior systemic therapy for MCC other than first-line aPD1 monotherapy (ie, chemotherapy) * Pregnancy or breastfeeding * Adverse events due to prior cancer therapy which are grade 3 or higher and have not resolved °Patients with prior grade 3 or higher immune related adverse events are not eligible, even if they have resolved * Prior severe hypersensitivity reaction (CTCAE version 5.0 grade ≥3) to avelumab * Prior radiotherapy which precludes the ability to safely deliver comprehensive ablative radiation therapy in the opinion of the treating radiation oncologist and principal investigator °Institutional guidelines for reirradiation will be used when making this determination * Known central nervous system metastases * Known clinically significant cardiovascular disease, defined as: * Stroke or myocardial infarction within 6 months of first dose of avelumab * Symptomatic congestive heart failure (New York Heart Association Class 2 or higher) * Serious arrhythmia requiring anti-arrhythmic agents * Known Human Immunodeficiency Virus infection * Known Hepatitis B or C infection requiring ongoing treatment * Vaccination within 4 weeks of first dose of avelumab °Inactivated vaccines are permissible * Iatrogenic immunosuppression with daily systemic corticosteroid equivalent of \>10 mg of prednisone * Active autoimmune disease that may cause clinical deterioration during immunotherapy °Including, but not limited to: * Inflammatory bowel disease or immune colitis * Immune mediated pneumonitis or pulmonary fibrosis * History of solid organ or hematopoietic transplant * Active infection requiring systemic therapy * Active suicidal ideation or behavior * Comorbid or diagnostic abnormalities which would interfere with interpretation of study results * Known hematopoietic cancer or dysfunction (i.e., leukemia, lymphoma) * Known non-MCC solid tumor with known metastasis or estimated risk of metastasis \>20% within 3 months