Avelumab With Binimetinib, Sacituzumab Govitecan, or Liposomal Doxorubicin in Treating Stage IV or Unresectable, Recurrent Triple Negative Breast Cancer

Inclusion Criteria: 1. Signed and dated written informed consent 2. Subjects \>= 18 years of age 3. Eastern Cooperative Oncology Group (ECOG) performance status 0 or 1 4. Clinical stage IV invasive breast cancer or unresectable locoregional recurrence of invasive breast cancer meeting the following criteria: * Estrogen receptor (ER)/progesterone receptor (PR)-negative (=\< 5% cells) by immunohistochemistry (IHC) and human epidermal grow (HER2) negative (by IHC or fluorescence in situ hybridization (FISH)) * Measurable disease as defined by Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 criteria and which can be followed by computed tomography (CT) or magnetic resonance imaging (MRI). A measurable lytic bone lesion(s) and/or skin lesion(s) are allowed. Skin lesions must also be followed by photography with measuring tools within the photograph at each tumor evaluation time point. Ultrasound may be used to follow breast lesions not visible by CT following discussion with Study Chair * Amenable to biopsy at the time of study entry * Known tumor/immune cell PD-L1 status by any assay 5. Adequate organ function including: * Cardiac ejection fraction at or above the institutional lower limit of normal, as assessed by either echocardiogram or multigated acquisition (MUGA) scan * Absolute neutrophil count (ANC) \>= 1.0 x 10\^9/L (may have received growth factor) * Platelets \>= 100 x 10\^9/L * Hemoglobin \>= 9 g/dL (may have been transfused) * Total serum bilirubin =\< 1.5 times upper limit of normal (ULN) * Aspartate aminotransferase (AST/serum glutamic oxaloacetic transaminase (SGOT)) and alanine aminotransferase (ALT/serum glutamate pyruvate transaminase (SGPT)) =\< 2.5 x ULN (or =\< 5 x ULN if liver metastases are present) * Serum creatinine =\< 1.5 x ULN or estimated creatinine clearance \>= 50 mL/min as calculated using the Cockcroft-Gault (CG) equation * Prothrombin time (PT)/international normalized ratio (INR) =\< 1.5 x ULN * Amylase =\< 1 x ULN testing is only required in patients with a history of pancreatic disorders (Abnormality not of pancreatic origin is allowed) * Participants with treated and controlled hypo or hyperthyroidism are eligible. 6. Male and female patients of childbearing potential must agree to use at least two methods of acceptable contraception from 15 days prior to first trial treatment administration until at least 30 days after study participant's final dose of study drug(s) \* NOTE: Females of childbearing potential are defined as those who are not surgically sterile or post-menopausal (i.e., patient has not had a bilateral tubal ligation, a bilateral oophorectomy, or a complete hysterectomy; or has not been amenorrhoeic for 12 months without an alternative medical cause). Post-menopausal status in females under 55 years of age should be confirmed with a serum follicle-stimulating hormone (FSH) level within laboratory reference range for postmenopausal women 7. Patients unable to read/write in English are eligible to participate in the overall study but will not participate in the Patient-Reported Outcome questionnaires throughout the trial. 8. Re-enrollment of a subject that has discontinued the study as a pre-randomization screen failure (i.e., a consented patient who was not randomized and did not receive any study treatment) is permitted. If re-enrolled, the subject must be re-consented. Only the screening procedures performed outside of protocol-specified timing must be repeated; if biopsies and correlative blood samples were already obtained, and patient has not received any systemic anti-cancer therapy since they were obtained, they do not need to be repeated. Exclusion Criteria: 1. More than 2 lines of chemotherapy in the metastatic setting 2. More than 1 prior line of checkpoint inhibitor therapy in the metastatic setting 3. Prior treatment with sacituzumab, govitecan 4. Concurrent anticancer therapy. Required washout from prior therapies are as follows: * Chemotherapy: \>= 14 days: antibody drug conjugants administered every 3 weeks require a 3-week washout. * Major surgery: \>=14 days (provided wound healing is adequate) * Radiation: \>= 7 days * Investigational/biologic therapy (half-life =\< 40 hours): \>= 14 days * Investigational/biologic therapy (half-life \> 40 hours): \>= 28 days * Use of corticosteroids or immunosuppressive medication is exclusionary, except the following in the absence of active autoimmune disease: * Subjects are permitted the use of corticosteroids with minimal systemic absorption (e.g. topical, ocular, intra-articular, intranasal, and inhaled) * Systemic corticosteroids at physiologic doses =\< 10 mg/day of prednisone or equivalent are permitted * Adrenal replacement steroid doses including doses \> 10 mg daily prednisone are permitted * A brief (less than 3 weeks) course of corticosteroids for prophylaxis (e.g. CT scan premedication against contrast dye allergy) or for treatment of nonautoimmune conditions (e.g. delayed-type hypersensitivity reaction caused by a contact allergen) is permitted 5. Previous malignant disease other than breast cancer within the last 5 years, with the exception of basal or squamous cell carcinoma of the skin, cervical carcinoma in situ, or low-risk cancers considered curatively treated (i.e. complete remission achieved at least 2 years prior to first dose of study drugs AND additional therapy not required while receiving study treatment) 6. All subjects with central nervous system metastases and/or carcinomatous meningitis, except those meeting the following criteria:: * Brain metastases that have been treated locally and are clinically stable for at least 2 weeks prior to enrollment * No ongoing neurological symptoms that are related to the brain localization of the disease (sequelae that are a consequence of the treatment of the brain metastases are acceptable) * Subjects must be either off steroids or on a stable or decreasing dose of =\< 10 mg daily prednisone (or equivalent) 7. Receipt of any organ transplantation including allogeneic stem-cell transplantation 8. Significant acute or chronic infections including, among others: * Known history of testing positive for human immunodeficiency virus (HIV), or acquired immunodeficiency syndrome (AIDS); testing is not required for this protocol. * A history of a positive test for hepatitis B virus (HBV) surface antigen (and/or core antibody) and/or confirmatory hepatitis C virus (HCV) ribonucleic acid (RNA) (if anti-HCV antibody tested positive); testing is not required for this protocol 9. Active autoimmune disease with reasonable possibility of clinically significant deterioration when receiving an immunostimulatory agent, per investigator discretion: * Subjects requiring hormone replacement with corticosteroids are eligible if the steroids are administered only for the purpose of hormonal replacement and at doses =\< 10 mg or 10 mg equivalent prednisone per day * Administration of steroids through a route known to result in a minimal systemic exposure (topical, intranasal, intro-ocular, or inhalation) is acceptable 10. History of interstitial lung disease that is symptomatic or which may interfere with the detection or management of suspected drug-related pulmonary toxicity 11. Uncontrolled asthma (defined as having 3 or more of the following features of partially controlled asthma within 28 days prior to starting study treatment: Daytime symptoms more than twice per week, any limitation of activities, any nocturnal symptoms/awaking, need for reliever/rescue inhaler more than twice per week, or known lung function \[peak expiratory flow (PEF) or forced expiratory volume in 1 second (FEV1)\] without administration of a bronchodilator that is \< 80% predicted or personal best \[if known\]) 12. Current symptomatic congestive heart failure (New York Heart Association \> class II), unstable cardiac arrhythmia requiring therapy (e.g. medication or pacemaker), unstable angina (e.g. new, worsening or persistent chest discomfort), uncontrolled hypertension (systolic \> 160 mmHg or diastolic \> 100mmHg), or known cardiac ejection fraction below the lower limit of institutional normal. Or any of the following occurring within 6 months (180 days) prior to first dose of avelumab: Myocardial infarction, coronary/peripheral artery bypass graft, cerebrovascular accident or transient ischemic attack. (Use of antihypertensive medication to control blood pressure is allowed.) 13. Patients who have neuromuscular disorders that are associated with elevated creatine kinase (CK) 14. History of acute or chronic pancreatitis 15. History or current evidence of retinal vein occlusion (RVO), or current risk factors for RVO including uncontrolled glaucoma, ocular hypertension, history of hyperviscosity, or hypercoagulability syndromes (patients with a history of pulmonary embolism or deep vein thrombosis (DVT) are allowed on study if they are also on anticoagulation as noted in (16) below) ; history of retinal degenerative disease. 16. Requirement of anticoagulant therapy with oral vitamin K antagonists such as Coumadin (warfarin). Low-dose anticoagulants for the maintenance of patency in a central venous access device or the prevention of deep vein thrombosis or pulmonary embolism is allowed. Therapeutic use of low molecular weight heparin or factor Xa inhibitors are allowed provided patients are safely able to interrupt it prior to biopsy procedures. 17. Persisting toxicity related to prior therapy that has not reduced to grade 1 (National Cancer Institute Common Toxicity Criteria for Adverse Events (NCI CTCAE) version 5.0); however, alopecia and sensory neuropathy grade =\< 2 is acceptable 18. Known severe (grade \>= 3 NCI-CTCAE v5.0) hypersensitivity reactions to monoclonal antibodies, or history of anaphylaxis 19. Vaccination within 28 days of the first dose of study drugs and while on trial is prohibited, except for administration of inactivated vaccines (for example, inactivated influenza vaccine) 20. Pregnant or breastfeeding females 21. Known current alcohol or drug abuse 22. Prisoners or subjects who are involuntarily incarcerated 23. Known psychiatric condition, social circumstance, or other medical condition reasonably judged by the patient's study physician to unacceptably increase the risk of study participation; or to prohibit the understanding or rendering of informed consent or anticipated compliance with scheduled visits, treatment schedule, laboratory tests and other study requirements.