Testing the Combination of New Anti-cancer Drug Peposertib With Avelumab and Radiation Therapy for Advanced/Metastatic Solid Tumors and Hepatobiliary Malignancies

Inclusion Criteria: * PHASE 1: Patients must have a histologically confirmed metastatic or locally advanced unresectable solid tumor that has progressed on or after available standard of care therapy or for which no acceptable standard of care therapy exists, or in which the patient declines standard of care therapy (each patient that declines standard of care therapy will be documented in the case report form) * PHASE 2: Patients must have a histologically confirmed metastatic or locally advanced unresectable cholangiocarcinoma/gallbladder carcinoma that has progressed on gemcitabine, cisplatin, and durvalumab/pembrolizumab * Age \>= 18 years * Because no dosing or adverse event data are currently available on the use of peposertib (M3814) in combination with avelumab in patients \< 18 years of age * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 (Karnofsky \>= 60%) * Patients with at least 1 index lesion to irradiate for whom palliative radiation treatment is indicated (including but not limited to pain and/or symptom control, prevention of disease -related complications, and preservation of organ function). Lung and liver lesions are preferred, though alternate lesions may be considered after discussion with trial principal investigator (PI). Up to 2 lesions may be considered for irradiation provided at least 1 lesion will receive the study treatment of total of 60 Gy and all prescribed irradiation will be completed within the radiation window * Patients with at least 1 Response Evaluation Criteria in Solid Tumors (RECIST) measurable lesion (to be unirradiated) (defined as those accurately measured in at least one dimension, with the longest diameter to be recorded for non-nodal lesions and the shortest diameter for nodal lesions). Measurable is defined as at least 10 mm in longest diameter for solid tumors, at least 15 mm in shortest diameter for lymph nodes * Patients must be willing to undergo fresh biopsies at baseline (as opposed to using archival tissue), in the event their baseline tissue was obtained \> 12 months prior to study consent and/or they are randomized to the gamma H2AX, pNBS1 and pKAP1 IFA with beta CATN segmentation assay * Absolute neutrophil count (ANC) \>= 1,500/mcL * Platelet count \>= 100,000/mcL * Hemoglobin \>= 9.0 g/dL * Serum creatinine =\< 1.5 x upper limit of normal (ULN) OR calculated serum creatinine clearance (glomerular filtration rate \[GFR\] can be used in place of creatinine or creatinine clearance) \>= 60 mL/min for participants with creatinine levels \> 1.5 x institutional ULN * Calculate serum creatinine clearance using the standard Cockcroft-Gault formula * Serum total bilirubin =\< 1.5 x ULN or direct bilirubin =\< ULN for participants with total bilirubin \> 1.5 x ULN * Patients with known Gilbert disease with serum bilirubin level =\< 3 x ULN are eligible * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\]) and alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x ULN or =\< 5.0 x ULN for patients with hepatobiliary tumors/liver metastases * Albumin \>= 2.8 g/L * International normalized ratio (INR) or prothrombin time (PT) or activated partial thromboplastin time (aPTT) =\< 1.5 x ULN * This applies only to patients not receiving therapeutic anticoagulation; patients receiving therapeutic anticoagulation should be on a stable dose * Participants must have the ability to swallow and retain oral medication and not have any clinically significant gastrointestinal abnormalities that might alter absorption * Female patients of childbearing potential must have a negative urine or serum pregnancy test. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required. The effects of peposertib (M3814) and avelumab on the developing human fetus are unknown and there is the potential for teratogenic or abortifacient effects. For this reason, women and men of child-bearing potential must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study treatment, and for 6 months after completion of peposertib (M3814) and avelumab administration. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with peposertib (M3814) and avelumab, breastfeeding should be discontinued if the mother is treated with peposertib (M3814) and avelumab * Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity (IDMC) who have a close caregiver or legally authorized representative (LAR) and/or family member available will also be eligible Exclusion Criteria: * PHASE I: Patients who have received prior anti-CTLA-4, anti-PD-1, anti-PD-L1 or other immune checkpoint inhibitor therapeutic antibodies or pathway-targeting agents * PHASE II: Patients who have received prior anti-CTLA-4, anti-PD-1, anti-PD-L1 or other immune checkpoint inhibitor therapeutic antibodies or pathway-targeting agents with the following exceptions: * Patients who have only received previous durvalumab (anti-PD-L1) in combination with gemcitabine +/- cisplatin as part of first line therapy (TOPAZ-1 regimen) are eligible * Patients who have only received previous pembrolizumab (anti-PD-1) in combination with gemcitabine +/- cisplatin as part of first line therapy (KEYNOTE-966 regimen) are eligible * Patients who have had chemotherapy, definitive radiation, biological cancer therapy, or investigational agent/device within 21 days of first planned dose of study therapy (within 14 days for palliative radiation). Previously irradiated lesions may be re-irradiated provided there is disease progression in the irradiated lesion and the prescribed radiation dosage can safely be re- administered * Patients who have not recovered from adverse events due to prior anti-cancer therapy (i.e., have residual toxicities \> Common Terminology Criteria for Adverse Events \[CTCAE\] grade 1) with the exception of alopecia * Patients with untreated/uncontrolled central nervous system (CNS)/leptomeningeal disease. Patients with asymptomatic, treated CNS disease are eligible if the treating physician determines that immediate CNS specific treatment is not required and is unlikely to be required during the first cycle of therapy and the following criteria are met: * Radiographic demonstration of clinical stability upon the completion of CNS-directed therapy and no evidence of interim progression between the completion of CNS-directed therapy and the screening radiographic study done \>= 4 weeks from completion of radiotherapy and \>= 2 weeks from discontinuation of corticosteroids * No stereotactic radiation or whole-brain radiation within 28 days prior to randomization * Patients with active autoimmune disease requiring systemic corticosteroids greater than the equivalent of prednisone 10 mg daily including but not limited to: systemic lupus erythematosus, rheumatoid arthritis, inflammatory bowel disease, colitis, vascular thrombosis associated with antiphospholipid syndrome, Wegener's granulomatosis, Sjogren's syndrome, Bell's palsy, Guillain-Barre syndrome, multiple sclerosis, autoimmune thyroid disease, vasculitis, or glomerulonephritis, with the following exceptions: * Patients with a history of autoimmune hypothyroidism on a stable dose of thyroid replacement hormone are eligible * Patients with controlled type 1 diabetes mellitus on a stable insulin regimen are eligible * Patients with eczema, psoriasis, lichen simplex chronicus of vitiligo with dermatologic manifestations only who require only low potency topical steroids (e.g., hydrocortisone 2.5%, hydrocortisone butyrate 0.1%, fluocinolone 0.01%, desonide 0.05%, alclometasone dipropionate 0.05%) are eligible * Patients receiving treatment with systemic immunosuppressive medications (including, but not limited to, prednisone, cyclophosphamide, azathioprine, methotrexate, thalidomide, and anti-tumor necrosis factor \[TNF\] agents) within 6 weeks must discontinue these medications prior to starting peposertib (M3814) and avelumab, with the exception of: * Patients with active autoimmune disease managed with systemic corticosteroids less than the equivalent of prednisone 10 mg daily * Patients who have received acute, low dose, systemic immunosuppressant medications (e.g., a one-time dose of dexamethasone for nausea) * The use of inhaled corticosteroids and mineralocorticoids (e.g., fludrocortisone) for patients with orthostatic hypotension and adrenocortical insufficiency * Patients who have undergone prior solid organ or bone marrow transplant with the exception of patients with prior renal transplant for whom dialysis may be employed in the event of graft rejection * Patients with uncontrolled intercurrent illness (e.g., including but not limited to uncontrolled hypertension \[HTN\] \[systolic blood pressure (BP) \> 150, diastolic BP \> 100\], symptomatic congestive heart failure \[CHF\], unstable angina pectoris, ischemic myocardial infarction \[MI\] within 6 months, cardiac arrhythmia, transient ischemic attack \[TIA or cerebrovascular accident (CVA)\]) within 6 months * Patients with serious active infection (e.g. requiring hospitalization and/or intravenous \[IV\] antibiotics) within 4 weeks prior to starting peposertib (M3814) and avelumab, or signs/symptoms of infection or receiving oral or IV antibiotics for the treatment of active systemic infection within 2 weeks prior to starting peposertib (M3814) and avelumab. Patients receiving prophylactic antibiotics are eligible * Patients with known chronic hepatitis B virus (HBV) infection must have an undetectable viral load on suppressive therapy if indicated. Patients with known chronic hepatitis C (HCV) infection must have been treated and cured. Patients who are currently on curative treatment are eligible if they have an undetectable HCV viral load * Patients with known human immunodeficiency virus (HIV) are allowed on study provided they have: * A stable regimen of highly active anti-retroviral therapy (HAART) * No requirement for concurrent antibiotics or antifungal agents for the prevention of opportunistic infection * A CD4 count above 250 cells/mcL * An undetectable HIV viral load on standard polymerase chain reaction (PCR)-based testing * Patients with history of idiopathic pulmonary fibrosis, pneumonitis (including drug induced), organizing pneumonia (e.g., bronchiolitis obliterans, cryptogenic organizing pneumonia), or evidence of active pneumonitis on screening chest computed tomography (CT) scan * Patients with known concurrent malignancy that is expected to require active treatment within two years, or may interfere with the interpretation of the efficacy and safety outcomes of this study in the opinion of the treating investigator. Superficial bladder cancer, nonmelanoma skin cancers, and low-grade prostate cancer not requiring cytotoxic therapy should not exclude participation in this trial. Patients with chronic lymphocytic leukemia (CLL) may be enrolled if they do not require active chemotherapy and their hematologic, renal and hepatic function meets criteria previously mentioned * Patients with psychiatric illness/social situations that would limit compliance with study requirements * History of allergic reactions attributed to compounds of similar chemical or biologic composition to peposertib (M3814) or avelumab * Patients unable to discontinue medications or substances that are potent inhibitors, inducers or sensitive substrates of CYP3A4/5 or CYP2C19 prior to starting peposertib (M3814) and avelumab are ineligible. Medications or substances that are strong inhibitors of CYP3A4/5 or CYP2C19 must be discontinued at least 1 week prior to first peposertib (M3814) dose. Strong inducers of CYP3A4/5 or CYP2C19 must be stopped at least 3 weeks prior to the first dose. Drugs mainly metabolized by CYP3A with a narrow therapeutic index as judged by the investigator must stop at least 1 day prior to first peposertib (M3814) dose. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product. The primary elimination mechanism of avelumab is proteolytic degradation, thus there are no contraindicated medications with respect to avelumab * Patients who cannot discontinue concomitant proton-pump inhibitors (PPIs) prior to starting peposertib (M3814) and avelumab. These must be discontinued \>= 5 days prior to starting peposertib (M3814) and avelumab. Patients do not need to discontinue calcium carbonate. H2 blockers are allowed provided they are taken at least 2 hours after peposertib (M3814) dose * Patients receiving sorivudine or any chemically related analogues (such as brivudine) and not able to discontinue prior to starting peposertib (M3814) and avelumab are excluded * Pregnant and lactating women are excluded from this study because peposertib (M3814) and avelumab are agents with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with peposertib (M3814) and avelumab, breastfeeding should be discontinued if the mother is treated with peposertib (M3814) and avelumab * Patients who have received live vaccination within 30 days before starting peposertib (M3814) and avelumab