Nivolumab Alone or in Combination With Ipilimumab in Treating Patients With Advanced Uterine Leiomyosarcoma

Inclusion Criteria: * Patients must have histologically or cytologically confirmed advanced leiomyosarcoma of the uterus (ULMS); advanced ULMS is defined as metastatic ULMS or unresectable primary ULMS * Patients must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for non-nodal lesions and short axis for nodal lesions) as \>= 20 mm (\>= 2 cm) with conventional techniques or as \>= 10 mm (\>= 1 cm) with spiral computed tomography (CT) scan, magnetic resonance imaging (MRI), or calipers by clinical exam * Patients must have received at least one prior line of chemotherapy, for ULMS (either in the adjuvant or metastatic setting) * Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1 * Life expectancy of greater than 9 months * Absolute neutrophil count \>= 1,500/mcL * Platelets \>= 100,000/mcL * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) (except patients with Gilbert syndrome, who can have total bilirubin \< 3.0 mg/dL) * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x ULN/ =\< 5 x ULN for subjects with liver metastases * Serum creatinine =\< 1.5 x ULN OR creatinine clearance (CrCl) \>= 50 mL/min (if using the Cockcroft-Gault formula) * Patients with a requirement for steroid treatment or other immunosuppressive treatment: patients should be excluded if they have a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study drug administration; inhaled or topical steroids and adrenal replacement doses \> 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease * Women of child-bearing potential (WOCBP) must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation; WOCBP should use an adequate method to avoid pregnancy for 31 weeks after the last dose of investigational drug; women of childbearing potential must have a negative serum or urine pregnancy test (minimum sensitivity 25 IU/L or equivalent units of human chorionic gonadotropin \[HCG\]) within 24 hours prior to the start of nivolumab; women must not be breastfeeding; women who are not of childbearing potential (i.e., who are postmenopausal or surgically sterile) do not require contraception * Women of childbearing potential (WOCBP) is defined as any female who has experienced menarche and who has not undergone surgical sterilization (hysterectomy or bilateral oophorectomy) or who is not postmenopausal; menopause is defined clinically as 12 months of amenorrhea in a woman over 45 in the absence of other biological or physiological causes; in addition, women under the age of 55 must have a documented serum follicle stimulating hormone (FSH) level less than 40 mIU/mL * WOCBP receiving nivolumab will be instructed to adhere to contraception for a period of 31 weeks after the last dose of investigational product; these durations have been calculated using the upper limit of the half-life for nivolumab (25 days) and are based on the protocol requirement that WOCBP use contraception for 5 half-lives plus 30 days * Should a woman become pregnant or suspect she is pregnant while participating in this study, she should inform the treating physician immediately * Ability to understand and the willingness to sign a written informed consent document * For enrollment in the first stage of Cohort B, patients must have accessible pre-treatment and post-treatment (4-6 weeks) tumor for biopsy Exclusion Criteria: * Patients who have had chemotherapy or radiotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events (AEs) due to agents administered more than 3 weeks earlier; patients who have had prior pelvic radiation may be at increased risk for bowel perforation, and therefore may not have residual inflammatory disease of the bowel or residual bowel toxicity based on baseline imaging and clinical assessment; palliative (limited-field) radiation therapy is permitted, if all of the following criteria are met: * Repeat imaging demonstrates no new sites of bone metastases * The lesion being considered for palliative radiation is not a target lesion * Bowel toxicity is not expected from the target field due to increased risk of perforation * Patients who are receiving any other investigational agents * Patients are excluded if they have had prior treatment with an anti-PD-1, anti-PD-L1, anti-PD-ligand 2 (L2), anti-cytotoxic T-lymphocyte-associated protein 4 (CTLA-4) antibody, or any other antibody or drug specifically targeting T-cell co-stimulation or immune checkpoint pathways * Active brain metastasis or leptomeningeal disease; patients with known brain metastases are allowed if metastases have been treated and there is no magnetic resonance imaging (MRI) evidence of progression for at least 12 weeks after treatment is complete and within 28 days prior to the first dose of nivolumab administration; there must also be no requirement for immunosuppressive doses of systemic corticosteroids (\> 10 mg/day prednisone equivalents) for at least 2 weeks prior to study drug administration * History of allergic reactions attributed to compounds of similar chemical or biologic composition to nivolumab * History of severe hypersensitivity reaction to any monoclonal antibody * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Pregnant women are excluded from this study; breastfeeding should be discontinued if the mother is treated with nivolumab * Patients should be excluded if they have known history of testing positive for human immunodeficiency virus (HIV) or known acquired immunodeficiency syndrome (AIDS) or if they have a positive test for hepatitis B virus surface antigen (HBV sAg) or hepatitis C virus ribonucleic acid (HCV antibody) indicating acute or chronic infection * Patients with active autoimmune disease or history of autoimmune disease that might recur, which may affect vital organ function or require immune suppressive treatment including systemic corticosteroids, are excluded; these include but are not limited to patients with a history of immune related neurologic disease, multiple sclerosis, autoimmune (demyelinating) neuropathy, Guillain-Barre syndrome, myasthenia gravis; systemic autoimmune disease such as systemic lupus erythematosus (SLE), connective tissue diseases, scleroderma, inflammatory bowel disease (IBD), Crohn's, ulcerative colitis, hepatitis; and patients with a history of toxic epidermal necrolysis (TEN), Stevens-Johnson syndrome, or phospholipid syndrome should be excluded; patients with vitiligo, endocrine deficiencies including thyroiditis managed with replacement hormones including physiologic corticosteroids are eligible; patients with rheumatoid arthritis and other arthropathies, Sjogren's syndrome and psoriasis controlled with topical medication and patients with positive serology, such as antinuclear antibodies (ANA), anti-thyroid antibodies should be evaluated for the presence of target organ involvement and potential need for systemic treatment but should otherwise be eligible * Patients are permitted to enroll if they have vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement, psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger (precipitating event) * Patients should be excluded if they have a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study drug administration; inhaled or topical steroids and adrenal replacement doses =\< 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease; patients are permitted to use topical, ocular, intra-articular, intranasal, and inhalational corticosteroids (with minimal systemic absorption); physiologic replacement doses of systemic corticosteroids are permitted, even if =\< 10 mg/day prednisone equivalents; a brief course of corticosteroids for prophylaxis (e.g., contrast dye allergy) or for treatment of non-autoimmune conditions (e.g., delayed-type hypersensitivity reaction caused by contact allergen) is permitted * Patients who have had evidence of active or acute diverticulitis, intra-abdominal abscess, gastrointestinal (GI) obstruction, fistula and abdominal carcinomatosis should be evaluated for the potential need for additional treatment before coming on study