Nivolumab and Ipilimumab and Stereotactic Body Radiation Therapy in Treating Patients With Salivary Gland Cancers

Inclusion Criteria: * Histologically proven salivary gland carcinoma (World Health Organization \[WHO\], 2005) arising from a previous head and neck primary site, and located within the head and neck region, lung or bone, and who are not candidates for curative intent therapy * Demonstrated disease progression during, or after discontinuation, of the most recent line of systemic therapy. For patients who have received no prior systemic therapy, demonstrated progression in the 3 months prior to trial participation assessed by the treating physician * Have received any number lines of prior systemic therapy (including systemic therapy in the curative intent setting) * Have a lesion/s deemed suitable by the treating physicians for stereotactic body radiation therapy (SBRT) with the intent of palliation or prevention of symptoms. This lesion must be: * 1-3 non-overlapping sites in the head and neck region OR * Metastatic lesions outside the head and neck (H\&N) region in the lung or bone (a minimum of 1 and a maximum 5 lesions will be irradiated), provided there is no significant overlap between the lesions \*\* Patients should have RECIST 1.1 criteria measurable disease in addition to the lesion/s treated with SBRT. If the site/s of SBRT were previously radiated to high dose radiation therapy (RT) (\> 50Gy), there should be \> 6 month time interval between the last dose of radiation and the start of SBRT * Have the ability to tolerate required SBRT-related procedures (e.g.: lie flat and hold position for treatment) as determined by the treating physician * Be willing and able to provide written informed consent for the trial and comply with the study visit requirements * Have measurable disease based on RECIST 1.1. (in addition to the lesion/s that will be treated with stereotactic radiation therapy) * Have provided tissue from an archival tissue sample or newly obtained core or excisional biopsy of a tumor lesion. Tissue requirement will be waived if deemed contraindicated or not clinically available/accessible for resection per the treating physician (principal investigator \[PI\] approval required) * Have a performance status of 0 or 1 on the Eastern Cooperative Oncology Group (ECOG) performance scale * Hemoglobin \>= 9.0 g/dL (performed within 28 days of treatment initiation) * Absolute neutrophil count (ANC) \>= 1.5 x 10\^9 /L (\>= 1500 per mm\^3) (performed within 28 days of treatment initiation) * Platelet count \>= 100 x 10\^9 /L (\>= 100,000 per mm\^3) (performed within 28 days of treatment initiation) * Serum bilirubin =\< 1.5 x institutional upper limit of normal (ULN). This will not apply to subjects with confirmed Gilbert's syndrome (persistent or recurrent hyperbilirubinemia that is predominantly unconjugated in the absence of hemolysis or hepatic pathology), who will be allowed only in consultation with their physician (performed within 28 days of treatment initiation) * Aspartate aminotransferases (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal unless liver metastases are present, in which case it must be =\< 5 x ULN (performed within 28 days of treatment initiation) * Serum creatinine clearance (CL) \> 40 mL/min by the Cockcroft-Gault formula (Cockcroft and Gault 1976) or by 24-hour urine collection for determination of creatinine clearance (performed within 28 days of treatment initiation) * Evidence of post-menopausal status OR negative urinary or serum pregnancy test for female pre-menopausal patients. Women will be considered post-menopausal if they have been amenorrheic for 12 months without an alternative medical cause. The following age-specific requirements apply: * Women \< 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of exogenous hormonal treatments and if they have luteinizing hormone and follicle stimulating hormone levels in the post-menopausal range for the institution or underwent surgical sterilization (bilateral oophorectomy, or hysterectomy) * Women \>= 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of all exogenous hormonal treatments, had radiation-induced menopause with last menses \> 1 year ago, had chemotherapy-induced menopause with last menses \> 1 year ago, or underwent surgical sterilization (bilateral oophorectomy, bilateral salpingectomy or hysterectomy) * Female subjects of childbearing potential should have a negative urine or serum pregnancy within 72 hours prior to receiving the first dose of study medication. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required * Female subjects of childbearing potential should be willing to use 1 method of highly effective birth control or be surgically sterile, or abstain from heterosexual activity for the course of the study through 180 days after the last dose of study medication. Subjects of childbearing potential are those who have not been surgically sterilized or have not been free from menses for \> 1 year * Male subjects should agree to use an adequate method of contraception starting with the first dose of study therapy through 180 days after the last dose of study therapy * Patient is \>= 5 years free of another primary malignancy, except: * If the other malignancy is basal cell carcinoma or cervical carcinoma in situ or * If the other primary malignancy is not considered clinically significant and is requiring no active intervention Exclusion Criteria: * Is currently participating in or has participated in a study of an investigational agent or using an investigational device within 4 weeks of the first dose of treatment * Has a target lesion/s for SBRT that demonstrate any of the following: * Located within 2 cm of the proximal bronchial tree * \> 5 cm (\> 50 cc) in greatest dimension * Has a target lesion/s in a region that previously received high dose radiation therapy (RT) (\> 50 Gy) demonstrating any of the following: * Carotid artery encasement (\> 180 degrees) (due to risk of carotid blow out) * Unprotected carotid artery (i.e. skin is directly over the carotid without intervening soft tissue, especially after prior neck dissection without a vascularized free flap) (due to risk of carotid blow out) * Skin infiltration by tumor (due to risk of fistula) * Located in the larynx/hypopharynx primaries (due airway threat) * Treated with high dose radiation therapy (\> 50 Gy) within 6 months or less of trial enrollment * Prior receipt of an anti-PD-1, anti-PDL1 or anti-CTLA4 immune checkpoint inhibitor * Current or prior use of immunosuppressive medication within 14 days before the first dose of nivolumab or ipilimumab. The following are exceptions to this criterion: * Intranasal, inhaled, topical steroids, or local steroid injections (e.g., intra articular injection) * Systemic corticosteroids at physiologic doses not to exceed 10 mg/day of prednisone or its equivalent * Steroids as premedication for hypersensitivity reactions (e.g., computed tomography \[CT\] scan premedication) * Has received a prior monoclonal antibody within 4 weeks prior to study Day 1 or who has not recovered (i.e., =\< Grade 1 or at baseline) from adverse events due to agents administered more than 4 weeks earlier * Has received prior chemotherapy, targeted small molecule therapy, or radiation therapy within 2 weeks prior to study Day 1 or who has not recovered (i.e., =\< Grade 1 or at baseline) from adverse events due to a previously administered agent. * Note: Subjects with =\< Grade 2 neuropathy are an exception to this criterion and may qualify for the study * Note: If subject received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting therapy * Has a known additional malignancy that is progressing or requires active treatment. Exceptions include basal cell carcinoma of the skin, squamous cell carcinoma of the skin, or in situ cervical cancer that has undergone potentially curative therapy * Has known brain metastases or spinal cord compression unless the patient is stable (asymptomatic; no evidence of new or emerging brain metastases; and stable and off steroids for at least 14 days prior to start of study treatment). Following radiotherapy and/or surgery of the brain metastases patients must wait 4 weeks following the intervention and before initiating study treatment with imaging to confirm stability * Has an active autoimmune disease requiring systemic treatment within the past 2 years or a documented history of clinically severe autoimmune disease, or a syndrome that requires systemic steroids or immunosuppressive agents. Subjects with vitiligo or resolved childhood asthma/atopy would be an exception to this rule. Subjects that require intermittent use of bronchodilators or local steroid injections would not be excluded from the study. Subjects with hypothyroidism stable on hormone replacement will not be excluded from the study * Has a history of or evidence of active interstitial lung disease or non-infectious pneumonitis * Has an active infection requiring systemic therapy * Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the subject's participation for the full duration of the trial, or is not in the best interest of the subject to participate, in the opinion of the treating investigator * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial * Is pregnant or breastfeeding, or expecting to conceive or father children within the projected duration of the trial, starting with the pre-screening or screening visit through 180 days after the last dose of trial treatment * Has a known history of human immunodeficiency virus (HIV) (HIV 1/2 antibodies) * Has evidence of acute or chronic hepatitis B, or hepatitis C * Has received a live vaccine within 30 days prior to the first dose of trial treatment * Has a history of primary immunodeficiency or an allogeneic organ transplant * Known history of previous clinical diagnosis of tuberculosis * Uncontrolled intercurrent illness including, but not limited to symptomatic congestive heart failure, uncontrolled hypertension, unstable angina pectoris, cardiac arrhythmia, active peptic ulcer disease or gastritis, seizures