Testing the Addition of an Anti-cancer Drug, BAY 1895344, to Usual Chemotherapy for Advanced Stage Solid Tumors, With a Specific Focus on Patients With Small Cell Lung Cancer, Poorly Differentiated Neuroendocrine Cancer, and Pancreatic Cancer

Inclusion Criteria: * DOSE ESCALATION COHORTS: Patients must have a biopsy-proven solid tumor that is metastatic or unresectable and has progressed on at least one line of standard therapy * DOSE ESCALATION COHORTS: Patients must have a solid tumor for which irinotecan or topotecan is considered standard of care * DOSE EXPANSION COHORTS: Patients must have biopsy proven metastatic or unresectable small cell lung cancer (SCLC), poorly differentiated neuroendocrine carcinoma (PD-NEC) (any extrapulmonary neuroendocrine carcinoma with small cell or large cell histology) or pancreatic adenocarcinoma (PDA) and have progressed on at least one line of standard therapy * DOSE EXPANSION COHORTS: Patients must have at least one measurable lesion outside of the lesion to be biopsied * Patients must be able to swallow pills * Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of BAY 1895344 in combination with irinotecan or topotecan in patients \< 18 years of age, children are excluded from this study * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 (Karnofsky \>= 60%) * Hemoglobin \> 9 g/dL * Absolute neutrophil count \>= 1,500/mcL * Platelets \>= 100,000/mcL * Total bilirubin =\< 2 x institutional upper limit of normal (ULN) * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x institutional ULN (=\< 5 x institutional ULN if liver metastases present) * Glomerular filtration rate (GFR) \>= 60 mL/min/1.73 m\^2 * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression. Furthermore, these patients must be asymptomatic from previously treated brain metastases (e.g. not on steroids for neurologic symptoms within 30 days of study enrollment) * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * Patients with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, patients should be class 2B or better * The effects of BAY 1895344 on the developing human fetus are unknown. For this reason and because DNA-damage response inhibitors as well as other therapeutic agents used in this trial are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation and for 6 months after completion of BAY 1895344 administration. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 6 months after completion of BAY 1895344 administration * Patient must have the ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity (IDMC) who have a legally-authorized representative (LAR) and/or family member available will also be eligible Exclusion Criteria: * Patients who have previously been treated with irinotecan will not be eligible to participate in the irinotecan arm and patients who have previously been treated with topotecan will not be eligible to participate in the topotecan arm. However, patients who previously received irinotecan may be treated with topotecan (and vice versa) should the other agent be considered a possible standard of care for their disease. Patients who have previously been treated with BAY 1895344 will be excluded from the study * Patients who have had chemotherapy or radiotherapy within 4 weeks (6 weeks for nitrosoureas or mitomycin C) prior to entering the study * Patients who have not recovered from adverse events due to prior anti-cancer therapy (i.e., have residual toxicities \> grade 1) with the exception of alopecia and endocrinopathies from prior immunotherapy * Patients who are receiving any other investigational agents * The investigator(s) must state a medical or scientific reason if patients who have brain metastases will be excluded from the study * History of allergic reactions attributed to compounds of similar chemical or biologic composition to BAY 1895344 or other agents used in study * Patients receiving any medications or substances that are substrates of CYP3A4 with a narrow therapeutic window, or strong inhibitors/inducers of CYP3A4 are ineligible, if they cannot be transferred to alternative medication. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Patients with uncontrolled intercurrent illness * Patients with psychiatric illness/social situations that would limit compliance with study requirements * Pregnant women are excluded from this study because BAY 1895344 is agent with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with BAY 1895344, breastfeeding should be discontinued if the mother is treated with BAY 1895344. These potential risks may also apply to other agents used in this study * Patients with an uncontrolled infection requiring IV antibiotics will not be eligible to participate in the study * Patients on strong CYP3A4 inhibitors must discontinue them at least 1 week prior to starting irinotecan therapy