Testing the Safety and Efficacy of the Combination of Two Anti-cancer Drugs, ZEN003694 and Abemaciclib, for Adult and Pediatric Patients (12-17 Years) With Metastatic or Unresectable NUT Carcinoma, Breast Cancer and Other Solid Tumors

Inclusion Criteria: * Participants must have histologically confirmed malignancy that is metastatic or unresectable and for which standard curative or palliative measures do not exist or are no longer effective * Dose Escalation Cohort Only: Participants must have evaluable disease or measurable disease per RECIST 1.1 criteria * Dose Expansion Cohort Only: * Participants must have a diagnosis of NUT carcinoma (NC) based on standard criteria for the disease, with diagnostic testing performed in a Clinical Laboratory Improvement Act (CLIA) certified laboratory: * Ectopic expression of NUT protein per World Health Organization (WHO) criteria as determined by immunohistochemistry (IHC) testing, OR * Detection of the NUT gene translocation as determined by fluorescence in situ hybridization (FISH) testing * Detection of the NUT gene translocation as determined by either deoxyribonucleic acid (DNA) next-generation sequencing (NGS) or ribonucleic acid (RNA) sequencing. * Participants must have measurable disease per RECIST 1.1 criteria * Any number of prior lines of therapy in the metastatic setting are allowed, including prior BET inhibitor therapy and prior CDK4/6 inhibitor therapy * Patients who received chemotherapy must have recovered (Common Terminology Criteria for Adverse Events \[CTCAE\] grade =\< 1) from the acute effects of chemotherapy except for residual alopecia or grade 2 peripheral neuropathy * Patients who received radiotherapy must have completed and fully recovered from the acute effects of radiotherapy. A washout period of at least 14 days is required between end of radiotherapy * Participants may have previously undergone surgical resection * Age \>= 12 years. Patients 12-17 years of age must be \> 40 kg at enrollment. Patients 12-17 years of age will not participate in the mandatory tumor biopsies. Since there is no data on patients less than 18 years of age, this population may require lower doses and additional safety precautions and should be closely monitored. Because no dosing or adverse event data are currently available on the use of ZEN003694 in combination with abemaciclib in patients \<12 years of age, younger children are excluded from this study * Eastern Cooperative Oncology Group (ECOG) performance status =\< 2 for participants \>= 16 years of age, Lansky \>= 50% if \< 16 years of age * Hemoglobin \>= 8 g/dL; Patients may receive erythrocyte transfusions to achieve this hemoglobin level at the discretion of the investigator. Initial treatment must not begin earlier than the day after the erythrocyte transfusion * Absolute neutrophil count \>= 1.5 x 10\^9/L * Platelets \>= 1 x 10\^11/L * Total bilirubin =\< 1.5 x institutional upper limit of normal (ULN) for age. Patients with Gilbert's syndrome with a total bilirubin =\< 2.0 times ULN and direct bilirubin within normal limits are permitted * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional ULN for age * Serum or plasma creatinine =\< 1.5 x institutional ULN OR calculated creatinine clearance \>= 50 mL/min (via the chronic kidney disease epidemiology (CKD-EPI) glomerular filtration rate estimation for participants \>= 18 years old, or 60 mL/min/1.73m\^2 for patients 12-17 years as calculated based on bedside Schwartz formula) * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load. Hepatitis C (HepC antibody) testing is required. Hepatitis C RNA is optional; however, a confirmatory negative Hepatitis C RNA test must be obtained to be able to enroll participants with positive Hepatitis C antibody due to prior resolved disease * Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression and has been clinically stable for at least 1 month. Patients must meet the following criteria: * Disease outside the CNS is present * Recovery from acute toxicity associated with the treatment to =\< CTCAE grade 1 or baseline (with the exception of alopecia), with no requirement for escalating doses of corticosteroids over the past 7 days * Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * Patients should be New York Heart Association Functional Classification of class 2B or better * Ability to swallow and retain oral medications * The effects of ZEN00364 and abemaciclib on the developing human fetus are unknown. For this reason and because BETi and CDKi-inhibiting agents are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. Women of child-bearing potential must have a negative pregnancy test prior to study entry. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men and women treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 3 weeks after completion of ZEN003694 and abemaciclib administration * For female subjects of child-bearing potentially receiving ZEN003694, hormonal means of birth control alone, such as oral, injectable, dermal, subdermal or topical contraceptives are NOT acceptable forms of birth control given that their efficacy has not been evaluated when given in combination with the investigational drugs. "Adequate contraception" is defined as the following: * Contraceptive methods with a failure rate of =\< 1% used in combination with the barrier method. The following contraceptive methods are acceptable to use in combination with the barrier method: intrauterine device (IUD), intrauterine system (IUS), or oral contraceptive pills (OCPs) that meet the \< 1% failure rate as stated in the product label. Note: Hormonal IUDs/OCPs may only be used if the following criteria are met: male condoms are required AND subjects are informed of the potential for reduced systemic hormone levels from the IUD/OCP when taking ZEN003694. Alternatively, male partner sterilization (vasectomy with documentation of azoospermia) prior to the female subject's entry into the study, and this male is the sole partner for that subject. For this definition, "documented" refers to the outcome of the investigator's/designee's medical examination of the subject or review of the subject's medical history for study eligibility, as obtained via a verbal interview with the subject or from the subject's medical records * Male subjects with female partners of child-bearing potential must use one of the following contraceptive methods: * Vasectomy with documentation of azoospermia OR * Condom use PLUS partner use of a highly effective contraceptive (=\< 1% rate of failure per year) such as intrauterine device or system, or hormonal birth control such as contraceptive subdermal implant, combined estrogen and progestogen oral contraceptive, injectable progestogen, contraceptive vaginal ring, or percutaneous contraceptive patches * Male subjects should not donate sperm while on study and for 12 weeks after the last dose of study medication. Male subjects whose partners are or become pregnant must continue to use condoms for 12 weeks after the last dose of study medication * Women of childbearing potential must have a negative pregnancy test within 7 days of starting treatment * Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity who have a legally-authorized representative (LAR) and/or family member available may be eligible after discussion with the Principal Investigator of this study. There will be a separate assent process for minors Exclusion Criteria: * Participants who have had cytotoxic chemotherapy, immunotherapy, or other investigational therapy within 2 weeks prior to entering the study. There is a two week required washout period for previous BET inhibitor therapy * Participants who have had radiotherapy within at least 2 weeks prior to entering the study. Stereotactic radiosurgery (SRS) within 1 week prior to entering the study will be allowed * Participants who have had major surgery within 3 weeks prior to entering the study * Participants who have received tyrosine kinase inhibitors (TKIs) or small molecules within 5 half-lives or 1 week (whichever is shorter) of study entry * Patients who are receiving any other investigational agents * History of allergic reactions attributed to compounds of similar chemical or biologic composition to ZEN003694 or abemaciclib * Patients requiring medications or substances that are strong inhibitors or strong inducers of CYP3A4 or CYP3A enzymes are ineligible. Strong inhibitors or inducers of CYP3A4 must be discontinued at least 7 days prior to the first dose of ZEN003694 and abemaciclib. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated list such as http://medicine.iupui.edu/clinpharm/ddis/table.aspx; medical reference texts such as the Physicians' Desk Reference may also provide this information. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Patients with uncontrolled intercurrent illness, including but not limited to: ongoing or active infection requiring systemic therapy, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, interstitial lung disease, severe dyspnea at rest or requiring oxygen therapy, severe renal impairment (e.g. estimated creatinine clearance \< 30ml/min), history of major surgical resection involving the stomach or small bowel, or preexisting Crohn's disease or ulcerative colitis or a preexisting chronic condition resulting in baseline grade 2 or higher diarrhea that, in the judgment of the investigator, would preclude participation in this study * Patients receiving any medications or substances that are strong inhibitors or inducers of CYP3A4 or substrates of CYP1A2 with narrow therapeutic windows are ineligible. Strong inhibitors or inducers of CYP3A4 must be discontinued at least 7 days prior to the first dose of ZEN003694. As proton pump inhibitors (PPIs), H2 receptor antagonists, and antacids may alter the pharmacokinetics of ZEN003694 by reducing ZEN003694 exposure, patients receiving proton pump inhibitors are ineligible. If H2 blockers or other acid reducing agents are used concomitantly with ZEN003694, a staggered dosing schedule should be used. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Pregnant women are excluded from this study because ZEN003694 is a BETi agent with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with ZEN003694, breastfeeding should be discontinued if the mother is treated with ZEN003694. These potential risks may also apply to other agents used in this study * Fridericia's correction formula (QTcF) \>= 450 msec on screening electrocardiogram (ECG) (machine or manual read allowed). Patients should avoid medications which prolong the QT * Patients receiving any medications or substances that are Factor Xa inhibitors (i.e., rivaroxaban, apixaban, betrixaban, edoxaban otamixaban, letaxaban, eribaxaban) or Factor IIa inhibitors (i.e., dabigatran). Low molecular weight heparin is allowed * Patients with radiation to \> 25% of the bone marrow * Patients who have had a bone-targeted radionuclide within 6 weeks of the first dose of ZEN003694 * Myocardial infarction or unstable angina within 6 months prior to the first dose of ZEN003694 * Impairment of gastrointestinal function that may significantly alter the absorption of ZEN003694 and/or abemaciclib * The patient has a personal history of any of the following conditions: syncope of cardiovascular etiology, ventricular arrhythmia of pathological origin (including, but not limited to, ventricular tachycardia and ventricular fibrillation), or sudden cardiac arrest