Testing the Combination of the Anti-cancer Drugs Copanlisib, Olaparib, and MEDI4736 (Durvalumab) in Patients With Advanced Solid Tumors With Selected Mutations

Inclusion Criteria: * ELIGIBILITY CRITERIA FOR ENROLLMENT INTO STEPS 1, 2, AND 3 * Patients must have germline or somatic mutations in DDR genes: BARD1, BRCA1, BRCA2, BRIP1, FANCA, NBN, PALB2, RAD51, RAD51B, RAD51C, RAD51D; or actionable mutations in the PTEN gene, or hotspot mutations in the PIK3CA gene (E542, E545 or H1047 are accepted). Local testing in Clinical Laboratory Improvement Act (CLIA)-certified laboratory will be accepted. Only mutations that have been recognized as actionable by the MD Anderson Precision Oncology Decision Support (PODS) team will be accepted * Patients must have histologically confirmed malignancy that is metastatic or unresectable and for which standard curative or palliative measures do not exist or are no longer effective * Patients must have measurable disease by Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 * Patients must be \>= 3 weeks beyond treatment with any chemotherapy or \>= 4 weeks beyond treatment with other investigational therapy to include hormonal, biological, or targeted agents; or at least 5 half-lives from hormonal, biological, or targeted agents, whichever is shorter at the time of treatment initiation * Age \>= 18 years. Because no dosing or adverse event (AE) data are currently available on the use of copanlisib in combination with olaparib +/- durvalumab (MEDI4736) in patients \< 18 years of age, children are excluded from this study, but will be eligible for future pediatric trials * Patients with a mutation within both the DDR and PTEN/PIK3CA pathways will be assessed by the genomics Precision Oncology Decision Support group at MD Anderson, and the patient will be allocated to the PI3K or DDR expansion group deemed to be the main driver. If the actionability between the groups is deemed to be equivocal, then the patient will be allocated to the expansion cohort with fewer patients * Eastern Cooperative Oncology Group (ECOG) performance status (PS) =\< 1 (Karnofsky \>= 60%) * Hemoglobin \>= 10 g/dL with no blood transfusion in the past 28 days * Leukocytes \>= 3,000/mcL * Lipase =\< 1.5 x upper limit of normal (ULN) * Absolute neutrophil count \>= 1,500/mcL * Platelets \>= 140,000/mcL * Total bilirubin =\< 1.5 x institutional ULN * Serum bilirubin =\< 1.5 x institutional ULN * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional ULN, unless liver metastases are present in which case they must be =\< 5 x ULN * Activated partial thrombin time =\< 1.5 x ULN unless subject is receiving anticoagulant therapy as long as prothrombin time (PT) or partial thromboplastin time (PTT) is within the therapeutic range of intended use of anticoagulants * International normalized ratio =\< 1.5 x ULN * Glomerular filtration rate (GFR) \>= 51 mL/min, based on a 24-hour urine test for creatinine clearance or estimated using the Cockcroft-Gault equation * Left ventricular ejection fraction (LVEF) \>= 50% * Patients who are therapeutically treated with an agent such as warfarin or heparin will be allowed to participate provided that their medication dose and international normalized ratio (INR)/PTT is stable * Prophylactic antiemetics may be administered according to standard practice. The routine use of standard antiemetics, including 5-hydroxytryptamine type 3 (5-HT3) blockers, such as granisetron, ondansetron, or an equivalent agent, is allowed as needed. The use of corticosteroids as antiemetics prior to copanlisib administration will not be allowed * Postmenopausal or evidence of non-childbearing status, a negative urine or serum pregnancy test within 28 days of study treatment and confirmed prior to treatment on day 1. Postmenopausal is defined as: * Amenorrheic for 1 year or more following cessation of exogenous hormonal treatments * Luteinizing hormone (LH) and follicle stimulating hormone (FSH) levels in the postmenopausal range for women under 50 * Radiation-induced oophorectomy with last menses \> 1 year ago * Chemotherapy-induced menopause with \> 1 year interval since last menses * Surgical sterilization (bilateral oophorectomy or hysterectomy) * Women of child-bearing potential MUST have a negative serum or urine human chorionic gonadotropin (HCG) test unless prior tubal ligation (\>= 1 year before screening), total hysterectomy or menopause (defined as 12 consecutive months of amenorrhea). Patients should not become pregnant or breastfeed while on this study * Patients and their partners, if sexually active and of childbearing potential, must agree to the use of two highly effective forms of contraception in combination throughout the period of taking study treatment and for 6 months after last dose of study drug(s) to prevent pregnancy in the study patient or partner. Male patients should avoid donating sperm for 3 months following the last dose of olaparib * Human immunodeficiency virus (HIV)-infected (HIV 1/2 antibody-positive; HIV testing pre-study not required) patients may participate IF they meet all the following eligibility requirements: * They must be on an anti-retroviral regimen with evidence of at least two undetectable viral loads within the past 6 months on this same regimen; the most recent undetectable viral load must be within the past 12 weeks * They must have a CD4 count \>= 250 cells/mcL over the past 6 months on this same anti-retroviral regimen and must not have had a CD4 count \< 200 cells/ mcL over the past 2 years, unless it was deemed related to the cancer and/or chemotherapy-induced bone marrow suppression * For patients who have received chemotherapy in the past 6 months, a CD4 count \< 250 cells/mcL during chemotherapy is permitted as long as viral loads were undetectable during this same chemotherapy * They must have an undetectable viral load and a CD4 count \>= 250 cells/mcL within 7 days of enrollment * They must not be currently receiving prophylactic therapy for an opportunistic infection and must not have had an opportunistic infection within the past 6 months * Ability to understand and the willingness to sign a written informed consent document. Patients with impaired decision-making capacity (IDMC) must have a legally authorized representative or caregiver who gives such consent * Patient is willing and able to comply with the protocol for the duration of the study, including undergoing treatment and scheduled visits and examinations * Patients with a tumor that is readily accessible for biopsy * ELIGIBILITY CRITERIA FOR ENROLLMENT INTO STEP 2 AND 3 ONLY * Body weight \> 30 kg * Life expectancy \>= 16 weeks Exclusion Criteria: * EXCLUSION CRITERIA FOR STEPS 1, 2, AND 3 * Persistent toxicities (\> Common Terminology Criteria for Adverse Events \[CTCAE\] grade 2) caused by previous cancer therapy, excluding alopecia * Patients receiving any systemic chemotherapy or radiotherapy (except for palliative reasons) within 3 weeks prior to study treatment * Major surgery within 2 weeks of starting study treatment and patients must have recovered from any effects of any major surgery * Patients who are receiving any other investigational agents * Other malignancy unless curatively treated with no evidence of disease for \>= 5 years except: adequately treated non-melanoma skin cancer, curatively treated in situ cancer of the cervix, ductal carcinoma in situ (DCIS), or stage 1, grade 1 endometrial carcinoma. A patient with a history of localized triple negative breast cancer may be eligible, provided the patient completed the adjuvant chemotherapy \> 3 years prior to registration, and the patient remains free of recurrent or metastatic disease * Patients with myelodysplastic syndrome/acute myeloid leukemia (MDS/AML) or with bone marrow findings consistent with MDS/AML * History of allergic reactions attributed to compounds of similar chemical or biologic composition to olaparib, copanlisib, PI3K inhibitors or MEDI4736 (durvalumab) or any of the excipients of any study products * Concomitant use of strong CYP3A inhibitors and inducers * Olaparib: concomitant use of known strong CYP3A inhibitors (e.g., itraconazole, telithromycin, clarithromycin, protease inhibitors boosted with ritonavir or cobicistat, indinavir, saquinavir, nelfinavir, boceprevir, telaprevir) or moderate CYP3A inhibitors (e.g., ciprofloxacin, erythromycin, diltiazem, fluconazole, verapamil). The required washout period for strong or moderate CYP3A inhibitors prior to starting olaparib is 2 weeks * Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Concomitant use of known strong CYP3A inducers (e.g., phenobarbital, enzalutamide, phenytoin, rifampicin, rifabutin, rifapentine, carbamazepine, nevirapine and St John's Wort) or moderate CYP3A inducers (e.g., bosentan, efavirenz, modafinil). The required washout period for strong or moderate CYP3A inducers prior to starting olaparib is 5 weeks for enzalutamide or phenobarbital and 3 weeks for other agents * Copanlisib: copanlisib is primarily metabolized by CYP3A4. Therefore, the concomitant use of strong inhibitors of CYP3A4 (e.g., ketoconazole, itraconazole, clarithromycin, ritonavir, indinavir, nelfinavir and saquinavir), and strong inducers of CYP3A4 (e.g. rifampin, phenytoin, carbamazepine, phenobarbital, St. John's Wort) are not permitted from 14 days prior to enrollment until the end of the study * Other medications that are prohibited while on copanlisib treatment: * Herbal medications/preparations (except for vitamins) * Anti-arrhythmic therapy other than beta blockers or digoxin * Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference for a list of drugs to avoid or minimize use of. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, uncontrolled major seizure disorder, unstable spinal cord compression, superior vena cava syndrome, extensive interstitial bilateral lung disease on high resolution computed tomography (HRCT) scan, symptomatic congestive heart failure, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Resting electrocardiogram (ECG) indicating uncontrolled, potentially reversible cardiac conditions, as judged by the investigator (e.g., unstable ischemia, uncontrolled symptomatic arrhythmia, congestive heart failure, QT corrected by the Fridericia formula \[QTcF\] prolongation of \> 500 msec, electrolyte disturbances), or patients with congenital long QT syndrome * Women who are breast feeding or pregnant are excluded from this study because olaparib is a poly (adenosine diphosphate-ribose) polymerase (PARP) inhibitor with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with olaparib, breastfeeding should be discontinued if the mother is treated with olaparib and copanlisib +/- MEDI4736 (durvalumab) * Previous allogenic bone marrow transplant or double umbilical cord blood transplantation (dUCBT) * Packed red blood cell or platelet transfusion in the last 28 days prior to study entry * Whole blood transfusions in the last 120 days prior to study entry. Whole blood transfusions performed within 120 days of study entry may interfere with blood samples taken for exploratory analysis * Patients should be excluded if they have a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of study drug administration. However, systemic corticosteroids may be indicated after starting the study drugs to treat immune-related adverse reactions. Inhaled or topical steroids and adrenal replacement doses =\< 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease * Patients with non-healing wound, ulcer, or bone fracture * Arterial or venous thrombotic or embolic events such as cerebrovascular accident (including transient ischemic attacks), deep vein thrombosis or pulmonary embolism within 6 months before the start of study medication * Patients with active, clinically serious infections \> grade 2 (CTCAE version\[v\] 5.0). Active infection including tuberculosis (clinical evaluation that includes clinical history, physical examination and radiographic findings, and tuberculosis \[TB\] testing in line with local practice), hepatitis B (known positive hepatitis B virus \[HBV\] surface antigen \[HBsAg\] result), or hepatitis C. Patients with a past or resolved HBV infection (defined as the presence of hepatitis B core antibody \[anti-HBc\] and absence of HBsAg) are eligible. Patients positive for hepatitis C (HCV) antibody are eligible only if polymerase chain reaction is negative for HCV ribonucleic acid (RNA). HBV/HCV screening is required 28 days prior to starting the study drug using a routine hepatitis virus lab panel * Active infection requiring IV antibiotics or other uncontrolled intercurrent illness requiring hospitalization * Patients unable to swallow orally administered medication and any medical condition or diagnosis that would likely impair absorption of an orally administered drug (e.g. gastrectomy, ileal bypass, chronic diarrhea, gastroparesis) * Known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial * Known active central nervous system (CNS) metastases and/or carcinomatous meningitis. Subjects with previously treated brain metastases may participate provided they are stable (without evidence of progression by imaging for at least four weeks prior to the first dose of trial treatment and any neurologic symptoms have returned to baseline), have no evidence of new or enlarging brain metastases, and are not using steroids for at least 7 days prior to trial treatment. This exception does not include carcinomatous meningitis which is excluded regardless of clinical stability * New York Heart Association class III or IV heart disease * History or concurrent interstitial lung disease of any severity and/or severely impaired lung function (as judged by the investigator) * Uncontrolled arterial hypertension despite optimal medical management (per investigator's opinion) * Patients with uncontrolled type I or II diabetes mellitus, defined as fasting blood glucose ˃ 160 mg/dL and glycosylated hemoglobin (HbA1c) ˃ 8%, are ineligible * EXCLUSION STEPS 2 AND 3 ONLY * Patients who have not recovered from grade \>= 2 adverse events due to prior anti-cancer therapy with the exception of alopecia, vitiligo, and the laboratory values defined in the inclusion criteria * Patients with grade \>= 2 neuropathy will be evaluated on a case-by-case basis after consultation with the study physician * Major surgical procedure (as defined by the investigator) within 28 days prior to the first dose of investigational product (IP). Note: local surgery of isolated lesions for palliative intent is acceptable * Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease \[e.g., colitis or Crohn's disease\], diverticulitis \[with the exception of diverticulosis\], systemic lupus erythematosus, Sarcoidosis syndrome, or Wegener syndrome \[granulomatosis with polyangiitis, Graves' disease, rheumatoid arthritis, hypophysitis, uveitis, etc.\]). The following are exceptions to this criterion: * Patients with vitiligo or alopecia * Patients with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement * Any chronic skin condition that does not require systemic therapy * Patients without active disease in the last 5 years may be included but only after consultation with the study physician * Patients with celiac disease controlled by diet alone * Receipt of live attenuated vaccine within 30 days prior to the first dose of IP. Note: Patients, if enrolled, should not receive live vaccine whilst receiving IP and up to 30 days after the last dose of IP * Patients with unstable angina pectoris * Patients who have received prior anti-PD-1, anti PD-L1 or anti CTLA-4: * Must not have experienced a toxicity that led to permanent discontinuation of prior immunotherapy. All adverse events (AEs) while receiving prior immunotherapy must have completely resolved or resolved to baseline prior to screening for this study * Must not have experienced a grade \>= 3 immune-related AE or an immune-related neurologic or ocular AE of any grade while receiving prior immunotherapy. Note: Patients with an endocrine AE of grade =\< 2 are permitted to enroll if they are stably maintained on appropriate replacement therapy and are asymptomatic * Must not have required the use of additional immunosuppression other than corticosteroids for the management of an AE, not have experienced recurrence of an AE if re-challenged, and not currently require maintenance doses of \> 10 mg prednisone or equivalent per day * Live vaccination, including virus vaccination and yellow fever vaccination, within 6 months before start of study treatment * Cytomegalovirus (CMV) infection. Patients who are known to be CMV polymerase chain reaction (PCR) positive at baseline will not be eligible