T-VEC With Chemotherapy or Endocrine Therapy in Treating Participants With HER2- Negative Breast Cancer

Inclusion Criteria: 1. Women or men 18 years or older with metastatic or locoregionally recurrent HER2-negative breast cancer. Resectable disease allowed. 2. Ability to understand and voluntarily sign informed consent prior to undergoing any study-related assessments or procedures, as well as adhere to the study visit schedule and other protocol requirements. 3. Histologic or cytologic confirmation of invasive breast cancer that is HER2-negative by standard clinical criteria. 4. Patients who will participate in the endocrine therapy cohort must have invasive breast cancer that is ER+ (\>=1% ER staining by IHC). 5. At least one accessible and injectable lesion (ie. breast, chest wall, skin nodule or mass, axillary or supraclavicular lymph node) of at least 1cm. (Ultrasound imaging may be used as clinically indicated. Injection must be able to be performed at the bedside). 6. Eastern Cooperative Oncology Group (ECOG) Performance Status of 0-1. 7. Concomitant use of bisphosphonates, RANKL antibody, and ovarian suppression is allowed. 8. Adequate organ function: * Absolute neutrophil count (ANC) \>= 1.5x109/L for chemotherapy cohort, and \>= 1.0x109/L for endocrine therapy cohort * Hemoglobin (Hgb) \>= 9g/dL * Platelets (plt) \>= 100 x 109/L for chemotherapy cohort, and \>=75, 000 for endocrine therapy cohort * aspartate aminotransferase (AST) and alanine aminotransferase (ALT) \<=2.5 x Upper Limit Normal (ULN). If liver metastases present \<=5 x ULN allowed. * Serum total bilirubin \<= 1.5 x ULN or direct bilirubin \<= 1.5 × ULN in patients with well documented Gilbert's Syndrome * Serum creatinine \<= 1.5 x ULN, or 24-hr clearance \>= 60ml/min * International normalization ratio (INR) or prothrombin time (PT) or partial thromboplastin time (PTT)/ activated partial thromboplastin time (aPTT) \<= 1.5 x ULN 9. Females of child-bearing potential (FCBP) should have a negative urine or serum pregnancy test within 72 hours prior to enrollment. If urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required. FCBP must also be willing to adhere to acceptable forms of birth control (a physician-approved contraceptive method: tubal ligation; intra-uterine device; barrier contraceptive with spermicide; or vasectomized partner) during the study treatment and through 3 months after the last dose of talimogene laherparepvec. FCBP are defined as sexually mature women who: * Have not undergone a hysterectomy (the surgical removal of the uterus) or bilateral oophorectomy (the surgical removal of both ovaries) or, * Have not been naturally postmenopausal for at least 12 consecutive months (i.e. has had menses at any time during the preceding 12 consecutive months) * Must be willing to practice abstinence or use effective contraception for a minimum of 3 months following completion of study treatment (in addition to during study therapy). Exclusion Criteria: 1. Any significant medical condition, laboratory abnormalities, which places the subject at unacceptable risk if he/she were to participate in the study. 2. Any condition that confounds the ability to interpret data from the study. 3. Patients must have recovered from side effects resulting from prior cancer-directed therapy to a level of grade 1 or less (unless deemed not clinically significant by study investigator). 4. Symptomatic central nervous system metastases. Subjects with brain metastases that have been previously treated and are stable for 4 weeks after whole-brain radiotherapy (WBXRT) or 2 weeks after Stereotactic radiosurgery (SRS) are allowed. Patients must be stable off steroids for brain metastases for at least 7 days. Subjects with asymptomatic clinically insignificant brain metastases not requiring treatment are allowed. The exception does not include carcinomatous meningitis which is excluded regardless of clinical stability. 5. Patients with leptomeningeal disease. 6. History of symptomatic autoimmune disease or active autoimmune disease that has required systemic treatment in the 2 weeks prior to enrollment. Replacement therapy (e.g., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment. 7. Evidence of immune suppression due to: a) known human immunodeficiency virus (HIV) infection or AIDS; b) known leukemia or lymphoma; c) those who require high dose steroids (\>10 mg/day of prednisone or equivalent within 7 days prior to enrollment) or other immunosuppressive therapies (\>2weeks); d) active hepatitis B or C; e) congenital or acquired cellular and/or humoral immune deficiency; f) other signs or symptoms of immune system suppression or concurrent opportunistic infection. 8. Nab-paclitaxel arm: Grade 2 or higher neuropathy. 9. Known history of: cardiac disease, heart failure or decreased left ventricular ejection fraction, significant clinical arrhythmias. 10. Patients must not have received an investigational agent within 4 weeks or \<= 5 half-lives, whichever is shorter, prior to starting study treatment. 11. Last dose of prior chemotherapy must be at least 2 weeks, and 2 weeks for targeted therapies, before first dose of study treatment. There is no required washout for endocrine therapy. 12. Major surgery or radiation ≤ 2 weeks prior to starting study treatment or who have not recovered from side effects of surgery or radiation. 13. Active herpetic skin lesions or prior complications of HSV-1 infection (e.g. herpetic encephalitis or keratitis). 14. Lesions with underlying infection or clinically meaningful bleeding. 15. Requires intermittent or chronic systemic (intravenous or oral) treatment with an antiherpetic drug (e.g. acyclovir), other than intermittent topical use. Patients requiring anti-herpetic prophylaxis during chemotherapy are excluded. 16. Previous treatment with talimogene laherparepvec or any other oncolytic virus. 17. Prior therapy with tumor vaccine. 18. Received live vaccine within 28 days prior to enrollment. 19. Known allergic reaction to talimogene laherparepvec, nab-paclitaxel, gemcitabine, carboplatin, aromatase inhibitors, tamoxifen, fulvestrant, or any of their components. An exception is made if the patient will not be receiving the offending agent/component (i.e. a patient who is allergic to nab-paclitaxel but will be receiving endocrine therapy is eligible). 20. Patients on therapeutic anticoagulation that cannot be held around injections. 21. Women who are pregnant or breast-feeding. 22. FCBP who are unwilling to use acceptable method(s) of effective contraception during study treatment and through 3 months after the last dose of talimogene laherparepvec. 23. Sexually active subjects and their partners unwilling to use a male or female latex condom to avoid potential viral transmission during sexual contact while on treatment and within 30 days after treatment with talimogene laherparepvec. 24. Subjects who are unwilling to minimize exposure with his/her blood or other body fluids to individuals who are at higher risks for HSV 1 induced complications (immunosuppressed individuals, HIV positive individuals, pregnant women, or children under the age of 1 year) during talimogene laherparepvec treatment and through 30 days after the last dose of talimogene laherparepvec.