Brachytherapy With Durvalumab or Tremelimumab for the Treatment of Patients With Platinum-Resistant, Refractory, Recurrent, or Metastatic Gynecological Malignancies

Inclusion Criteria: * Subjects with platinum-resistant or refractory, recurrent or metastatic gynecological malignancies, including ovarian, endometrial, or cervical cancer are eligible for enrollment. Subjects unsuitable or refusing platinum-based chemotherapy are allowed to enrolled * Measurable disease according to Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1 criteria. Thus, subjects with metastatic disease must have at least 1 lesion, not previously irradiated, that can be accurately measured at baseline as \>=10 mm in the longest diameter (except lymph nodes which must have a short axis ≥15 mm) with computed tomography (CT) or magnetic resonance imaging (MRI) and that is suitable for accurate repeated measurement as per RECIST v1.1 guidelines * Disease amenable to be treated safely with brachytherapy * Capable of giving signed informed consent which includes compliance with the requirements and restrictions listed in the informed consent form (ICF) and in this protocol * Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1 * Life expectancy of \>= 12 weeks * Subjects must consent to provide an archived tumor specimen from within 12 months prior to study entry (ie, from subject signing consent to participate in the study). If not available, subjects should have at least 1 lesion amenable to biopsy and consent to provide a pre-treatment fresh biopsy. Additional archival tissue from beyond 12 months prior to study entry is also requested, if available, to support exploratory analyses * Hemoglobin \>= 9.0 g/dL * Absolute neutrophil count (ANC) \>= 1.5 (or 1.0) x (\>= 1500 per mm\^3) * Platelet count \>= 100 (or 75) x 10\^9/L (\>= 75,000 per mm\^3) * Serum bilirubin =\< 1.5 x institutional upper limit of normal (ULN) * This will not apply to subjects with confirmed Gilbert's syndrome (persistent or recurrent hyperbilirubinemia that is predominantly unconjugated in the absence of hemolysis or hepatic pathology), who will be allowed only in consultation with their physician * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional upper limit of normal unless liver metastases are present, in which case it must be =\< 5 x ULN * Creatinine \< 3 x upper limit of normal * Evidence of post-menopausal status or negative urinary or serum pregnancy test for female pre-menopausal subjects. Women will be considered post-menopausal if they have been amenorrheic for 12 months without an alternative medical cause. The following age-specific requirements apply * Women \< 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of exogenous hormonal treatments and if they have luteinizing hormone and follicle-stimulating hormone levels in the post-menopausal range for the institution or underwent surgical sterilization (bilateral oophorectomy or hysterectomy) * Women \>= 50 years of age would be considered post-menopausal if they have been amenorrheic for 12 months or more following cessation of all exogenous hormonal treatments, had radiation-induced menopause with last menses \> 1 year ago, had chemotherapy-induced menopause with last menses \>1 year ago, or underwent surgical sterilization (bilateral oophorectomy, bilateral salpingectomy or hysterectomy) * Subject is willing and able to comply with the protocol for the duration of the study including undergoing treatment and scheduled visits and examinations including follow up Exclusion Criteria: * Prior use of checkpoint inhibitors * Prior radiation in which the 50% isodose line overlaps with intended site for brachytherapy * Radiation treatment to more than 30% of the bone marrow or with a wide field of radiation within 4 weeks of the first dose of study drug * Eastern Cooperative Oncology Group (ECOG) performance status of 2 or greater * Involvement in the planning and/or conduct of the study (applies to both AstraZeneca staff and/or staff at the study site) * Participation in another clinical study with an investigational therapeutic pharmaceutical agent i.e. chemotherapy, targeted therapy, or immunotherapy =\< 21 days or =\< 5 half-lives * Concurrent enrolment in another clinical study, unless it is an observational (non-interventional) clinical study or during the follow-up period of an interventional study * Receipt of the last dose of anticancer therapy (chemotherapy, immunotherapy, endocrine therapy, targeted therapy, biologic therapy, tumor embolization, monoclonal antibodies) =\< 21 days or 5 half-lives prior to the first dose of study drug. If sufficient wash-out time has not occurred due to the schedule or pharmacokinetics (PK) properties of an agent, a longer wash-out period will be required, as agreed by AstraZeneca/MedImmune and the investigator * Any unresolved toxicity National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) grade \>= 2 from previous anticancer therapy with the exception of alopecia, vitiligo, and the laboratory values defined in the inclusion criteria * Subjects with grade \>= 2 neuropathy will be evaluated on a case-by-case basis after consultation with the study physician * Subjects with irreversible toxicity not reasonably expected to be exacerbated by treatment with durvalumab or tremelimumab may be included only after consultation with the study physician * Any concurrent chemotherapy, investigation product (IP), biologic, or hormonal therapy for cancer treatment. Concurrent use of hormonal therapy for non-cancer-related conditions (e.g., hormone replacement therapy) is acceptable * Major surgical procedure (as defined by the investigator) within 28 days prior to the first dose of IP. Note: Local surgery of isolated lesions for palliative intent is acceptable * History of allogenic organ transplantation * Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease \[e.g., colitis or Crohn's disease\], diverticulitis \[with the exception of diverticulosis\], systemic lupus erythematosus, Sarcoidosis syndrome, or Wegener syndrome \[granulomatosis with polyangiitis, Graves' disease, rheumatoid arthritis, hypophysitis, uveitis, etc\]). The following are exceptions to this criterion: * Subjects with vitiligo or alopecia * Subjects with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement * Any chronic skin condition that does not require systemic therapy * Subjects without active disease in the last 5 years may be included but only after consultation with the study physician * Subjects with celiac disease controlled by diet alone * Uncontrolled intercurrent illness, including but not limited to, ongoing or active infection, symptomatic congestive heart failure, uncontrolled hypertension, unstable angina pectoris, cardiac arrhythmia, interstitial lung disease, serious chronic gastrointestinal conditions associated with diarrhea, or psychiatric illness/social situations that would limit compliance with study requirement, substantially increase risk of incurring adverse events (AEs) or compromise the ability of the subject to give written informed consent * History of another primary malignancy except for: * Malignancy treated with curative intent and with no known active disease \>= 5 years before the first dose of IP and of low potential risk for recurrence * Adequately treated non-melanoma skin cancer or lentigo maligna without evidence of disease * Adequately treated carcinoma in situ without evidence of disease * History of leptomeningeal carcinomatosis * Brain metastases or spinal cord compression. Subjects with suspected brain metastases at screening should have an MRI (preferred) or CT each preferably with IV contrast of the brain prior to study entry. Brain metastases will not be recorded as RECIST target lesions at baseline if study allows subjects with brain metastasis (mets) * Has untreated central nervous system (CNS) metastases and/or carcinomatous meningitis identified either on the baseline brain imaging (RECIST) obtained during the screening period or identified prior to signing the ICF. Subjects whose brain metastases have been treated may participate provided they show radiographic stability (defined as 2 brain images, both of which are obtained after treatment to the brain metastases. These imaging scans should both be obtained at least four weeks apart and show no evidence of intracranial progression). In addition, any neurologic symptoms that developed either as a result of the brain metastases or their treatment must have resolved or be stable either, without the use of steroids, or are stable on a steroid dose of =\< 10 mg/day of prednisone or its equivalent for at least 14 days prior to the start of treatment. Brain metastases will not be recorded as RECIST target lesions at baseline * Mean QT interval corrected for heart rate using Fridericia's formula (QTcF) \>= 470 ms calculated from 3 electrocardiogram (ECG)s (within 15 minutes at 5 minutes apart) * For durvalumab monotherapy and durvalumab + tremelimumab combination studies this criterion can be removed. For durvalumab +/- tremelimumab in combination with an agent with pro-arrhythmic potential or where effect of the combination on QT is not known if this criterion should be retained. Subject safety and the cardiac SKG should be consulted as needed * History of active primary immunodeficiency * Active infection including tuberculosis (clinical evaluation that includes clinical history, physical examination and radiographic findings, and tuberculosis (TB) testing in line with local practice), hepatitis B (known positive hepatitis B virus \[HBV\] surface antigen (HBsAg) result), hepatitis C, or human immunodeficiency virus (positive HIV 1/2 antibodies). Subjects with a past or resolved HBV infection (defined as the presence of hepatitis B core antibody \[anti-HBc\] and absence of HBsAg) are eligible. Subjects positive for hepatitis C (HCV) antibody are eligible only if polymerase chain reaction is negative for HCV ribonucleic acid (RNA) * Current or prior use of immunosuppressive medication within 14 days before the first dose of durvalumab or tremelimumab. The following are exceptions to this criterion: * Intranasal, inhaled, topical steroids, or local steroid injections (e.g., intra articular injection) * Systemic corticosteroids at physiologic doses not to exceed 10 mg/day of prednisone or its equivalent * Steroids as premedication for hypersensitivity reactions (e.g., CT scan premedication) * Receipt of live attenuated vaccine within 30 days prior to the first dose of IP. Note: Subjects, if enrolled, should not receive live vaccine whilst receiving IP and up to 30 days after the last dose of IP * Female subjects who are pregnant or breastfeeding or male or female subjects of reproductive potential who are not willing to employ effective birth control from screening to 90 days after the last dose of durvalumab monotherapy or180 days after the last dose of durvalumab + tremelimumab combination therapy * Known allergy or hypersensitivity to any of the study drugs or any of the study drug excipients * Prior randomization or treatment in a previous durvalumab and/or tremelimumab clinical study regardless of treatment arm assignment * Determined by the investigator that the subject is unsuitable to participate in the study and the subject is unlikely to comply with study procedures, restrictions and requirements * Known allergy or hypersensitivity to IP or any excipient