Testing the Safety of the Combination of Anti-Cancer Drugs CX-5461 (Pidnarulex) and Trastuzumab Deruxtecan (T-DXd) for Human Epidermal Growth Factor Receptor 2 (HER2)-Positive Solid Tumors and Breast Cancer

Inclusion Criteria: * DOSE ESCALATION PHASE ONLY: Patients must have histologically confirmed malignancy that is metastatic or unresectable and for which standard curative or palliative measures do not exist or are no longer effective * DOSE EXPANSION PHASE ONLY: Participants must have histologically or cytologically confirmed invasive breast cancer, with either locally advanced or metastatic disease * Age ≥ 18 years. Because no dosing or adverse event data are currently available on the use of CX-5461 (pidnarulex) in combination with T-DXd in patients \< 18 years of age, children are excluded from this study * Eastern Cooperative Oncology Group (ECOG) performance status ≤ 2 (Karnofsky ≥ 60%) * Absolute neutrophil count (ANC) ≥ 1,500/mcL * No administration of granulocyte colony-stimulating factor (G-CSF) is allowed within 1 week prior to screening assessment * Platelets ≥ 100,000/mcL * No transfusions with red blood cells or platelets are allowed within 1 week prior to screening assessment * Total bilirubin ≤ 1.5 × institutional upper limit of normal (ULN) (or ≤ 2 × institutional ULN in patients with documented Gilbert's syndrome) * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) ≤ 3 × institutional ULN (or ≤ 5 × ULN in patients with liver metastases) * International normalized ratio (INR)/prothrombin time (PT) and activated partial thromboplastin time (aPTT) ≤ 1.5 × ULN * Glomerular filtration rate (GFR) ≥ 60 mL/min/1.73 m\^2 (using the Cockcroft-Gault equation for participants with creatinine levels above institutional ULN) * Patients must have had at least one prior line of cytotoxic chemotherapy. Patients can have received an unlimited number of additional lines of chemotherapy, targeted therapy, biologic therapy, or hormonal therapy. Patients must not have progressed on a prior anthracycline in the metastatic setting. Receipt of anthracycline in the (neo)adjuvant setting is allowed, provided that disease recurrence occurred later than 6 months after the completion of treatment * Prior poly (ADP-ribose) polymerase (PARP) inhibition is allowed * No specific germline mutation is required * DOSE ESCALATION PHASE ONLY: * HER2-positive (IHC 3+) solid cancers, * HR+ HER2 positive/low/ultralow breast cancer or triple negative breast cancer (TNBC) HER2-low breast cancer, with HER2 positive defined by the current American Society of Clinical Oncology (ASCO)/College of American Pathologists (CAP) guidelines, * HER2-low, with HER2-low defined as IHC 2+/in situ hybridization (ISH)-, IHC 1+/ISH-, or IHC 1+/ISH untested * DOSE EXPANSION PHASE ONLY: * Either the primary invasive tumor and/or the metastasis must be: * HER2-low, with HER2-low defined as IHC 2+/ISH-, IHC 1+/ISH-, or IHC 1+/ISH untested. Any estrogen receptor (ER) and progesterone receptor (PR) expression is permitted but must be known, or * HR+ HER2-ultralow defined as IHC 0 with membrane staining * Participants must have at least one lesion that is not within a previously radiated field that is measurable per Response Evaluation Criteria in Solid Tumors (RECIST) version (v)1.1. Bone lesions are not considered measurable by definition. Biopsy of the lesion that will be used for disease evaluation (measurable disease) is not allowed in the dose expansion portion of this study * Peripheral neuropathy grade ≤ 1 * Patients must have left ventricular ejection fraction (LVEF) ≥ 50% by either an echocardiogram (ECHO) or multi-gated acquisition (MUGA) scan within 28 days before randomization/enrollment * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression * Patients with new or progressive brain metastases (active brain metastases) or leptomeningeal disease are eligible if the treating physician determines that immediate CNS specific treatment is not required and is unlikely to be required during the first cycle of therapy * Patients with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, patients should be class II or better * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * The effects of CX-5461 (pidnarulex) and T-DXd on the developing human fetus are unknown. For this reason, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control or abstinence) 14 days prior to study entry and for the duration of study participation and for at least 7 months (women of childbearing potential \[WOCBP\] only) after the last dose of study drug. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Women should not breastfeed while taking CX-5461 (pidnarulex) and T-DXd and for 7 months after cessation of treatment. Men treated or enrolled on this protocol must also agree to use adequate contraception 14 days prior to the study, for the duration of study participation, and 6 months after completion of CX-5461 (pidnarulex) and T-DXd administration * Women of non-child-bearing potential defined as pre-menopausal females with a documented tubal ligation or hysterectomy; or postmenopausal defined as 12 months of spontaneous amenorrhea (in questionable cases, a blood sample with simultaneous follicle-stimulating hormone \[FSH\] \> 40 mIU/mL and estradiol \< 40 pg/mL \[\< 147 pmol/L\] is confirmatory) or on ovarian suppression are eligible. Females on hormone replacement therapy (HRT) and whose menopausal status is in doubt will be required to use one of the contraception methods outlined for women of child-bearing potential if they wish to continue their HRT during the study. Otherwise, they must discontinue HRT to allow confirmation of post-menopausal status prior to study enrollment. For most forms of HRT, at least 2-4 weeks will elapse between the cessation of therapy and the blood draw; this interval depends on the type and dosage of HRT. Following confirmation of their post-menopausal status, they can resume use of HRT during the study without use of a contraceptive method * Male patients must not freeze or donate sperm starting at screening and throughout the study period, and at least 6 months after the final study drug administration. Preservation of sperm should be considered prior to enrollment in this study * Female patients must not donate, or retrieve for their own use, ova from the time of screening and throughout the study treatment period, and for at least 7 months after the final study drug administration * Ability to understand and the willingness to sign a written informed consent document. Legally authorized representatives (LAR) may sign and give informed consent on behalf of study participants * Patients who have had chest radiation therapy within 4 weeks (2 weeks for palliative stereotactic radiation therapy). These patients will be excluded because T-DXd is known to increase the risk of developing pneumonitis Exclusion Criteria: * Patients with a history of (non-infectious) interstitial lung disease (ILD) that required steroids, have current ILD, or where there is suspected ILD that cannot be ruled out by imaging at screening. These patients will be excluded because T-DXd is known to increase the risk of developing ILD and pneumonitis * Patients with clinically severe pulmonary compromise resulting from intercurrent pulmonary illnesses including, but not limited to, any underlying pulmonary disorder (i.e., pulmonary emboli within three months of the study enrollment, severe asthma, severe chronic obstructive pulmonary disease \[COPD\], restrictive lung disease, pleural effusion, etc.), and any autoimmune, connective tissue or inflammatory disorders with potential pulmonary involvement (i.e., Rheumatoid arthritis, Sjogren's, sarcoidosis, etc.), or prior pneumonectomy at the time of screening. These patients will be excluded because T-DXd is known to increase the risk of developing ILD and pneumonitis * Patients who have had chemotherapy (including antibody drug therapy, retinoid therapy, hormonal therapy for cancer) within 3 weeks (2 weeks or five half-lives, whichever is longer for small-molecule targeted agents such as 5-fluorouracil-based agents, folinate agents), weekly paclitaxel; 6 weeks for nitrosoureas or mitomycin C. These patients will be excluded to allow for recovery of toxicities related to chemotherapy and minimize risk of drug interactions * Patients who have had cancer immunotherapy including monoclonal antibody therapy within 4 weeks * Patients who have had a major surgery within 4 weeks * Patients with unresolved toxicities from previous anticancer therapy, defined as toxicities (other than alopecia) not yet resolved to grade ≤ 1 or baseline. Patients with chronic grade 2 toxicities may be eligible (e.g., grade 2 chemotherapy-induced neuropathy). Patients should no longer be symptomatic nor require treatment with corticosteroids or anticonvulsants and must have recovered from the acute toxic effect of radiotherapy * Eligibility of subjects receiving any medications or substances known to affect or with the potential to affect the activity of CX-5461 (pidnarulex) will be determined based on their potential to interact with the CYP3A4 isozyme. Specifically, subjects taking strong CYP3A4 inhibitors or strong CYP3A4 inducers will be excluded from participation in the trial. A list of agents that interact with CYP450 isoenzymes is provided * Patients who are receiving any other investigational agents * History of allergic reactions attributed to compounds of similar chemical or biologic composition to CX-5461 (pidnarulex), T-DXd, or the inactive ingredients in the drug products, including patients who have a history of severe hypersensitivity reactions to other monoclonal antibodies * Patients with a corrected QT interval (QTc) prolongation to \> 470 ms (females) or \> 450 ms (males) based on average of the screening triplicate 12-lead electrocardiogram (ECG) * Patients with clinically significant corneal disease, cicatricial conjunctivitis or active ocular surface disease in the opinion of the investigator * Patients with uncontrolled intercurrent illness or any other significant condition(s) that would make participation in this protocol unreasonably hazardous * Pregnant women are excluded from this study because CX-5461 (pidnarulex) and T-DXd are agents with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events (AEs) in nursing infants secondary to treatment of the mother with CX-5461 (pidnarulex) and T-DXd, breastfeeding should be discontinued if the mother is treated with CX-5461 (pidnarulex) and T-DXd and avoided for 7 months after the last dose