Talimogene Laherparepvec in Treating Patients With Recurrent Breast Cancer That Cannot Be Removed by Surgery

Inclusion Criteria: * Histological confirmation of breast carcinoma * Histological confirmation of recurrence of chest wall with or without distant metastasis disease * Patients must have failed at least 1 systemic regimen or have clinical stable disease with capecitabine, hormonal therapy (with or without mTOR inhibitor or CDK4/6 inhibitor), or anti HER-2 therapy (trastuzumab, pertuzumab, ado-trastuzumab emtansine, lapatinib) for at least 2 months after their diagnosis of locoregional/metastatic disease * Concurrent radiation therapy is permitted after the study treatment is initiated so long as the planned radiation field doesn't overlap with planned injection sites * Eastern cooperative oncology group performance status (ECOG PS) 0-1 * Absolute neutrophil count (ANC) \>= 1.0 x 10\^9/L; if patient is taking CDK4/6 inhibitor or capecitabine, there is a need for maintaining ANC \>= 1.0 consistently for at least 2 months without dose changes * Platelet count \>= 75 x 10\^9/L, if patient is taking CDK4/6 inhibitor, ado-trastuzumab emtansine or capecitabine, there is a need for maintaining platelet count \>= 75 x 10\^9/L without dose changes * Hemoglobin \>= 8.0 g/L * International normalization ratio (INR) or prothrombin time (PT) 1.5 x upper limit of normal (ULN), unless the subject is receiving anticoagulant therapy, in which case PT and partial thromboplastin time (PTT)/ activated PTT (aPTT) must be within therapeutic range of intended use of anticoagulants * Serum creatinine =\< 1.5 x upper limit of normal (ULN), OR 24-hour creatinine clearance \>= 60 mL/min for subject with creatinine levels \> 1.5 x ULN; (Note: creatinine clearance need not be determined if the baseline serum creatinine is =\< 1.5 x ULN; creatinine clearance should be determined per institutional standard) * Aspartate aminotransferase (AST) =\< 2.5 x ULN if liver metastases present * Alanine aminotransferase (ALT) =\< 2.5 x ULN if liver metastases present * Total bilirubin =\< 1.5 x ULN, OR direct bilirubin =\< ULN for a subject with total bilirubin level \> 1.5 x ULN * Subjects must be candidate for intralesional injection into cutaneous, subcutaneous or nodal tumors with or without image ultrasound guidance defined as one or more of the following at least 1 injectable lesion \>= 5 mm in longest diameter, multiple injectable lesions that in aggregate have a longest diameter of \>= 5 mm * Female patients of childbearing potential must have negative urine pregnancy test no more than 3 days prior to starting study treatment * Patients must be able and willing to give written informed consent Exclusion Criteria: * Patients who have operable disease with curable intent, and/or are candidates for radiation therapy for local control * Patients receiving concurrent anti-cancer therapy (chemotherapy except capecitabine or ado-trastuzumab emtansine, immunotherapy) while taking study medication, or have previously received talimogene laherparepvec or any other oncolytic virus * Patients with metastatic sites that requires chemotherapy (except capecitabine or ado-trastuzumab emtansine) * Known active central nervous metastases; subjects with previously treated brain metastases may participate provided they are stable (without evidence of progression by imaging for at least four weeks prior to the first dose of trial treatment and any neurologic symptoms have returned to baseline), have no evidence of new or enlarging brain metastases, and are not using steroids \> 10 mg/day pf prednisone or equivalent; the exception does not include carcinomatosis meningitis which is excluded regardless of clinical stability * More than three lesions per organ for visceral metastases except for lung or lymph node sites * History or evidence of active autoimmune disease that has required systemic treatment (i.e., use of corticosteroids, immunosuppressive drugs or disease modifying agents); replacement therapy (eg, thyroxine for hypothyroidism, insulin for diabetes or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency) is not considered a form of systemic treatment for autoimmune disease * Patients with concurrent disease or condition that would make them inappropriate for study participation, or any serious medical disorder that would interfere with patients' safety * History of other malignancy within the past 5 years with the following exceptions: a) Adequately treated non melanoma skin cancer without evidence of disease at the time of enrollment b). Adequately treated cervical carcinoma in situ without evidence of disease at the time of enrollment c). Adequately treated breast ductal carcinoma in situ without evidence of disease at the time of enrollment d). Prostatic intraepithelial neoplasia without evidence of prostate cancer at the time of enrollment * Patients with active infection and requiring intravenous (IV) or oral antibiotics * Evidence of immune suppression as following: * Known human immunodeficiency virus (HIV) infection or acquired immunodeficiency syndrome (AIDS) * Known leukemia or lymphoma * Primary immunodeficiency state such as severe combined immunodeficiency disease * Concurrent opportunistic infection * Receiving systemic immunosuppressive therapy (\> 2 weeks) including oral steroid doses \> 10 mg/day of prednisone or equivalent within 7 days prior to the initiation of study treatment * Known hepatitis B or C infection * Congenital or acquired cellular and/or humoral immune deficiency * Other signs or symptoms of immune system suppression * Active herpetic skin lesions or prior complication of herpes simplex virus (HSV)-1 infections (e.g. herpetic encephalitis or keratitis) * Currently pregnant or breast-feeding, or planning to become pregnant during study treatment and through 3 months after the last dose of study treatment * Female subject of childbearing potential who is unwilling to use acceptable method(s) of effective contraception during study treatment and through 3 months after the last dose of talimogene laherparepvec; (women of not childbearing potential: post-menopausal \[age \> 55 years with cessation of menses \> 12 months or \< 55 years but not spontaneous menses for at least 2 years or \< 55 years and spontaneous menses within the past 1 year, but currently amenorrheic (eg, spontaneous or secondary to hysterectomy), and with postmenopausal gonadotropin levels (luteinizing hormone and follicle-stimulating hormone levels \> 40 IU/L) or postmenopausal estradiol levels (\< 5 ng/dL) or according to the definition of "postmenopausal range" for the laboratory involved\] or who have had a hysterectomy, bilateral salpingectomy, or bilateral oophorectomy) * Sexually active subjects and their partners unwilling to use male or female latex condom to avoid potential viral transmission during sexual contact while on treatment and within 30 days after treatment with talimogene laherparepvec * Currently enrolled in another clinical trial for investigational drugs, procedures or device (excluding non-cancer treatment trials) or receipt of an investigational agent or device within 4 weeks of the initiation of study treatment * Requires intermittent or chronic treatment with antiherpetic drugs, except for topical agents * Patients who are known sensitive to any of the products or components to be administered during treatment with talimogene laherparepvec * Chronic oral or systemic steroid medication use at a dose of \> 10 mg/d of prednisone or equivalent (steroids with low systemic absorption \[e.g. triamcinolone hexacetonide\] injected into joint space are allowed) * Prior therapy with tumor vaccine, or received live vaccine within 28 days prior to the initiation of study treatment * Subjects who are unwilling to minimize exposure with his/her blood or other body fluids to individuals who are at higher risks for HSV-1 induced complications such as immunosuppressed individuals, individuals known to have HIV infection, pregnant women, or infants under the age of 3 months, during talimogene laherparepvec treatment and through 30 days after the last dose of talimogene laherparepvec * Prior immunosuppressive, chemotherapy, radiotherapy (in which the field encompassed a planned injection site), biological cancer therapy (monoclonal antibodies), or major surgery within 28 days prior to enrollment or has not recovered to Common Terminology Criteria for Adverse Events (CTCAE) grade 1 or better from adverse event due to cancer therapy administered more than 28 days prior to enrollment * Prior radiotherapy in which the field does not overlap the injection sites or non-immunosuppressive targeted therapy within 14 days prior to enrollment or has not recovered to CTCAE grade 1 or better from adverse event due to cancer therapy administered more than 14 days prior to enrollment * Patients who have extensive skin disease, defined as total area of skin involvement/ lesions that comprise \> 10% of body surface area; skin involvement comprising \> 5% to upper anterior chest wall or \> 5% to upper posterior back is excluded