Phase II Study of REPotrectinib With or Without Fulvestrant in Patients With Hormone Receptor-positive Human Epidermal Growth Factor 2-negative Metastatic Invasive LObular Carcinoma Who Received a Prior Endocrine Therapy in Combination With Cyclin-dependent Kinase 4 and 6 Inhibitor (REPLOT Trial)

Inclusion Criteria: * 18 years of age or older. * Confirmed ILC with negative E-cadherin immunohistochemistry (IHC) staining on pre-treatment biopsy or archival biopsy. o Participants with ILC who have a germline CDH1 mutation will be included if the E-cadherin IHC staining is negative. * Estrogen receptor positive (\>1%), progesterone receptor positive or negative, and HER2-negative according to HER2 testing guidelines from the American Society of Clinical Oncology/College of American Pathologists. * Patients must be willing to undergo biopsy as required by the study, if the tumor is safely accessible. * Participant must have been exposed to a CDK4/6i prior to enrollment. * Participants who received prior chemotherapy, ADCs, mTOR inhibitor and/or PI3K are eligible. o Participants with ESR1 mutation who received prior elacestrant can still enroll on the study. If they did not receive prior fulvestrant they will be enrolled on Cohort 1. If they received prior fulvestrant they will be enrolled on Cohort 2. * Participants should not have received more than 2 chemotherapeutic agents and/or ADCs in the metastatic setting. o The enrollment of patients who received 2 or more prior line of therapy (including endocrine therapy) in the metastatic setting will be limited to 50% of the total accrual in both cohorts. * Eastern Cooperative Oncology Group (ECOG) Performance status ≤2 * Participant has either measurable disease per response evaluation criteria in solid tumors (RECIST) v1.1 criteria OR at least one predominantly lytic bone lesion must be present. * Women of childbearing potential (WOCBP) must have a negative serum or urine pregnancy test within 2 weeks prior to study treatment initiation. * WOCBP must agree to use adequate contraception for the duration of study treatment and 7 months after the last dose of study treatment. * Participants must have adequate organ and marrow function as defined below: Absolute neutrophil count ≥1,000/mcL Platelets ≥70,000/mcL Total Bilirubin ≤ institutional upper limit of normal (ULN) AST(SGOT)/ALT(SGPT) ≤3 × institutional ULN Creatinine ≤ institutional ULN * For participants with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated. * Participants with a history of hepatitis C virus (HCV) infection must have been treated and cured. For participants with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load. * Participants with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression. * Participants with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial. * Participants with no current symptoms of cardiac disease. * Ability to understand and the willingness to sign a written informed consent document. * The effects of repotrectinib on the developing human fetus are unknown. For this reason, women of child-bearing potential must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. (Refer to Pregnancy Assessment Policy MD Anderson Institutional Policy # CLN1114). This includes all female participants, between the onset of menses (as early as 8 years of age) and 55 years unless the patient presents with an applicable exclusionary factor which may be one of the following: Postmenopausal (no menses in greater than or equal to 12 consecutive months). History of hysterectomy or bilateral salpingo-oophorectomy. Ovarian failure (Follicle Stimulating Hormone and Estradiol in menopausal range, who have received Whole Pelvic Radiation Therapy). History of bilateral tubal ligation or another surgical sterilization procedure. Approved methods of birth control are as follows: Hormonal contraception (i.e. birth control pills, injection, implant, transdermal patch, vaginal ring), Intrauterine device (IUD), Tubal Ligation or hysterectomy, Subject/Partner post vasectomy, Implantable or injectable contraceptives, and condoms plus spermicide. Not engaging in sexual activity for the total duration of the trial and the drug washout period is an acceptable practice; however periodic abstinence, the rhythm method, and the withdrawal method are not acceptable methods of birth control. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Exclusion Criteria: * Participants with symptomatic visceral disease or any disease burden that makes the participant ineligible for endocrine therapy per the investigator's best judgment. * Participants who have had chemotherapy, hormonal therapy, biotherapy, immunotherapy or radiotherapy within 4 weeks prior to entering the study. * Systemic small molecule-targeted therapies (eg, tyrosine kinase inhibitors) within 5 half-lives or 4 weeks (whichever is shorter) prior to the first dose of study drug. * Participants who have not recovered from adverse events due to prior anti-cancer therapy (i.e., have residual toxicities \> Grade 1) with the exception of alopecia. * Participants who are receiving any other investigational agents. * Participants with prior history of pneumonitis/ILD * Participants with Grade ≥2 ataxia, muscle weakness, dysgeusia, dizziness, paresthesia and/or peripheral neuropathy. * Participants with untreated or progressing brain metastases. * History of allergic reactions attributed to compounds of similar chemical or biologic composition to repotrectinib, fulvestrant or other agents used in study. * Participants with psychiatric illness/social situations that would limit compliance with study requirements. * Pregnant women are excluded from this study because repotrectinib and fulvestrant have the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with repotrectinib, breastfeeding should be discontinued if the mother is treated with repotrectinib. These potential risks may also apply to other agents used in this study.