Nivolumab With or Without Ipilimumab in Treating Patients With Persistent or Recurrent Epithelial Ovarian, Primary Peritoneal, or Fallopian Tube Cancer

Inclusion Criteria: * Patients must have recurrent or persistent epithelial ovarian, fallopian tube, or primary peritoneal cancer with documented disease progression (disease not amendable to curative therapy); histologic confirmation of the original primary tumor is required via the pathology report; NOTE: patients with mucinous histology are NOT eligible; patients with carcinosarcoma histology are NOT eligible * All patients must have measurable disease as defined by RECIST 1.1; measurable disease is defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded); each lesion must be \>= 10 mm when measured by computed tomography (CT), magnetic resonance imaging (MRI) or caliper measurement by clinical exam; or \>= 20 mm when measured by chest x-ray; lymph nodes must be \>= 15 mm in short axis when measured by CT or MRI * Patients must have at least one "target" lesion to be used to assess response on this protocol as defined by RECIST 1.1; tumors within a previously irradiated field will be designated as "non-target" lesions unless progression is documented or a biopsy is obtained to confirm persistence at least 90 days following completion of radiation therapy * Appropriate for study entry based on the following diagnostic workup: * History/physical examination within 28 days prior to registration * Imaging of target lesion(s) within 28 days prior to registration * Further protocol-specific assessments: * Recovery from effects of recent surgery, radiotherapy or chemotherapy * Free of active infection requiring antibiotics (with the exception of uncomplicated urinary tract infection \[UTI\]) * Any hormonal therapy directed at the malignant tumor must be discontinued at least one week prior to registration * Any other prior therapy directed at the malignant tumor including chemotherapy, targeted agents, immunologic agents, and any investigational agents, must be discontinued at least 4 weeks prior to registration (6 weeks for nitrosoureas or mitomycin C) * Any prior radiation therapy must be completed at least 4 weeks prior to registration * At least 4 weeks must have elapsed since major surgery * Patients are allowed to have received up to three prior cytotoxic regimens for treatment of their epithelial ovarian, fallopian tube, or primary peritoneal cancer; they must have had one prior platinum-based chemotherapeutic regimen for management of primary disease, possibly including intra-peritoneal therapy, consolidation, biologic/targeted (non-cytotoxic) agents or extended therapy (maintenance/consolidation) administered after surgical or non-surgical assessment; patients are allowed to have received, but are not required to have received, one to two cytotoxic regimens for management of recurrent or persistent disease; (for the purposes of this study poly adenosine diphosphate \[ADP\] ribose polymerase \[PARP\] inhibitors given for recurrent or progressive disease will be considered cytotoxic; PARP inhibitors given as maintenance therapy in continuation with management of primary disease will not be considered as a separate cytotoxic regimen); if two cytotoxic regimens had been received for management of recurrent or persistent disease, one of these regimens would have had to contain either a platinum or a taxane agent * Performance status of 0, 1 or 2 within 28 days prior to registration * Absolute neutrophil count (ANC) \>= 1,500/ul (within 14 days prior to registration) * Platelets \>= 100,000/ul (within 14 days prior to registration) * Creatinine =\< 1.5 x institutional/laboratory upper limit of normal (ULN) (within 14 days prior to registration) * Bilirubin =\< 1.5 x ULN; for patients with Gilbert's syndrome, bilirubin =\< 3.0 mg/dL is acceptable (within 14 days prior to registration) * Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) =\< 3 x ULN (within 14 days prior to registration) * Albumin \>= 2.8 g/dL (within 14 days prior to registration) * Adequate thyroid function within 28 days prior to registration defined as serum thyroid-stimulating hormone (TSH) in normal range * The patient or a legally authorized representative must provide study-specific informed consent prior to study entry * Platinum-free interval (PFI) - patients must have progressed \< 12 months after completion of their last platinum-based chemotherapy; the date (platinum free interval) should be calculated from the last administered dose of platinum therapy to documentation of progression * Adequate oxygen saturation via pulse oximeter within 28 days prior to registration (i.e., patient can NOT have CTCAE hypoxia grade 2 or greater) * Left ventricular ejection fraction (LVEF) \>= 50% (measured within 28 days of study entry) Exclusion Criteria: * Patients who have had prior therapy with nivolumab or with an anti-programmed cell death (PD)-1, anti-PD-ligand (L)1, anti-PD-L2, anti-cytotoxic T-lymphocyte-antigen (CTLA)-4 antibody, or any other antibody or drug specifically targeting T-cell co-stimulation or immune check point pathways * History of severe hypersensitivity reaction to any monoclonal antibody * Patients with a history of other invasive malignancies, with the exception of non-melanoma skin cancer and other specific malignancies are excluded if there is any evidence of other malignancy being present within the last three years (2 years for breast cancer); patients are also excluded if their previous cancer treatment contraindicates this protocol therapy * Patients who have received prior chemotherapy for any abdominal or pelvic tumor OTHER THAN for the treatment of ovarian, fallopian tube, or primary peritoneal cancer within the last three years are excluded; patients may have received prior adjuvant chemotherapy and radiotherapy for localized breast cancer, provided that it was completed more than 2 years prior to registration, and the patient remains free of recurrent or metastatic disease and hormonal therapy has been discontinued; patients who have received prior radiotherapy to any portion of the abdominal cavity or pelvis or thoracic cavity within the last three years are excluded; prior radiation for localized cancer of the head and neck or skin is permitted, provided that it was completed more than three years prior to registration, and the patient remains free of recurrent or metastatic disease * Patients with uncontrolled illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure and unstable angina pectoris * Patients with history of organ transplant * Patients who are pregnant or nursing; women of child-bearing potential (WOCBP) must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation; WOCBP should use an adequate method to avoid pregnancy for 23 weeks after the last dose of investigational drug; WOCBP must have a negative serum or urine pregnancy test (minimum sensitivity 25 IV/L or equivalent units of human chorionic gonadotropin \[HCG\]) within 24 hours prior to the start of nivolumab or nivolumab + ipilimumab; women must not be breastfeeding; women who are not of childbearing potential (i.e., who are postmenopausal or surgically sterile) do not require contraception; women of childbearing potential (WOCBP) is defined as any female who has experienced menarche and who has not undergone surgical sterilization (hysterectomy and/or bilateral oophorectomy) or who is not postmenopausal; menopause is defined clinically as 12 month amenorrhea in a woman over 45 in the absence of other biological or physiological causes; in addition, women under the age of 55 must have a documented serum follicle stimulating hormone (FSH) level greater than 40 mIU/mL; if, following initiation of the investigational product(s), it is subsequently discovered that a study subject is pregnant or may have been pregnant at the time of investigational product exposure, including during at least 6 half-lives after product administration, the investigational product will be permanently discontinued in an appropriate manner (e.g., dose tapering if necessary for subject safety); the investigator must report this event and any outcomes by amendment through Cancer Therapy Evaluation Program (CTEP)-Adverse Event Reporting System (AERS); protocol-required procedures for study discontinuation and follow-up must be performed on the subject unless contraindicated by pregnancy (e.g., X-ray studies); other appropriate pregnancy follow-up procedures should be considered if indicated; in addition, the investigator must report and follow-up on information regarding the course of the pregnancy, including perinatal and neonatal outcome; infants should be followed for a minimum of 8 weeks * History or evidence upon physical examination of central nervous system (CNS) disease, including primary brain tumor, seizures which are not controlled with non-enzyme inducing anticonvulsants, any brain metastases and/or epidural disease, or history of cerebrovascular accident (cerebrovascular accident \[CVA\], stroke), transient ischemic attack (TIA) or subarachnoid hemorrhage within six months prior to the first date of study treatment * In order for patients with known history of testing positive for human immunodeficiency virus (HIV) or known acquired immunodeficiency syndrome (AIDS) to be eligible, they must be on a stable highly active antiretroviral therapy (HAART) regimen, have cluster of differentiation (CD)4 counts \> 350, with no detectable viral load on quantitative polymerase chain reaction (PCR) * Patients with treated hepatitis virus infections (hepatitis B or hepatitis C) are eligible if they have been definitively treated for 6 months, have no detectable viral load on quantitative PCR, and liver function tests (LFTs) meet eligibility requirements * Patients with active autoimmune disease or history of autoimmune disease that might recur, which may affect vital organ function or require immune suppressive treatment including systemic corticosteroids, should be excluded; these include but are not limited to patients with a history of immune related neurologic disease, multiple sclerosis, autoimmune (demyelinating) neuropathy, Guillain-Barre syndrome, myasthenia gravis; systemic autoimmune disease such as systemic lupus erythematosus (SLE), connective tissue diseases, scleroderma, inflammatory bowel disease, Crohn's, ulcerative colitis, hepatitis; and patients with a history of toxic epidermal necrolysis (TEN), Stevens-Johnson syndrome, or phospholipid syndrome should be excluded; patient with vitiligo, endocrine deficiencies including thyroiditis managed with replacement hormones including physiologic corticosteroids are eligible; patients with rheumatoid arthritis and other arthropathies, Sjogren's syndrome and psoriasis controlled with topical medication and patients with positive serology, such as antinuclear antibodies (ANA), anti-thyroid antibodies should be evaluated for the presence of target organ involvement and potential need for systemic treatment but should otherwise be eligible * Patients are permitted to enroll if they have vitiligo, type I diabetes mellitus, residual hypothyroidism due to autoimmune condition only requiring hormone replacement (such as Hashimoto's thyroiditis), psoriasis not requiring systemic treatment, or conditions not expected to recur in the absence of an external trigger (precipitating event) * Patients should be excluded if they have a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study drug administration; inhaled or topical steroids and adrenal replacement doses =\< 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease; patients are permitted to use topical, ocular, intra-articular, intranasal, and inhalational corticosteroids (with minimal systemic absorption); a brief course of corticosteroids for prophylaxis (e.g., contrast dye allergy) or for treatment of non-autoimmune conditions (e.g., delayed-type hypersensitivity reaction caused by contact allergen) is permitted * Any of the following within 2 months of registration: active peptic ulcer disease, diverticulitis, cholecystitis, symptomatic cholangitis or appendicitis, malabsorption syndrome; any of the following within 6 months of registration: intra-abdominal abscess, gastrointestinal obstruction requiring parenteral hydration and/or nutrition, gastrointestinal perforation; Note: complete resolution of an intra-abdominal abscess must be confirmed prior to registration even if the abscess occurred more than 6 months prior to registration * No planned concomitant, non-protocol directed anti-cancer therapy * Grade \>= 2 peripheral neuropathy