Goserelin and Letrozole or Anastrozole in Premenopausal Patients With Stage II-III Estrogen Receptor-Positive Breast Cancer

Inclusion Criteria: * Patient must have histological or cytological confirmed invasive breast cancer * Patient must be premenopausal confirmed by serum estradiol level in the premenopausal range (\> 25 pg/ml) at the beginning of the study; for women on oral contraceptives, these agents must be held for two weeks before the estradiol assessment is made * Patient must have a negative serum pregnancy test within 7 days of registration * Patient's tumor must be ER+ with or without concomitant progesterone receptor-positive (PR+) with an Allred score of 6, 7 or 8; patients with \> 66.6% of cells staining positive by conventional immunohistochemistry (IHC) have a minimum Allred score of 6 and are eligible * Patient's tumors must be HER2 negative by local laboratory assessment: HER2 IHC 0, 1+, or 2+ with subsequent negative fluorescent in situ hybridization (FISH) (ratio \< 1.8); negative FISH alone in absence of IHC is acceptable * Patient must have T2-T4c, any N, M0 breast cancer, by clinical staging (physical examination) * Patient's primary tumor must be palpable and measure \> 2 cm by tape, ruler or caliper measurements in at least one dimension * Patient must have mammogram and ultrasound of the breast within 42 days prior to registration; if a patient has clinically palpable or suspicious nodes, then an ultrasound of the axilla is also required * Patient, as documented by the treating physician, must be clinically staged as one of the following: * T4 a-c for which modified radical mastectomy with negative margins is the goal * T2 or T3 for which conversion from needing mastectomy to breast conservation is the goal * T2 for which lumpectomy at first attempt is the goal * Patient must be \> or = 18 years old. * Patient must stop taking all forms of hormonal treatment, including oral or other form of hormonal contraceptive methods and all forms of hormone replacement therapy, at least two weeks prior to starting protocol therapy * Patient must agree to use a "highly-effective form of non-hormonal contraception" (applies to patient and/or partner) * Patient must be willing to undergo oophorectomy, if indicated * Patient must have an Eastern Cooperative Oncology Group (ECOG) performance status of =\< 2 * Patient must have normal organ and marrow function as defined below: * Leukocytes \>= 3,000/mcL * Absolute neutrophil count \>= 1,500/mcL * Platelets \>= 100,000/mcL * Total bilirubin within normal institutional limits * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamic pyruvate transaminase \[SGPT\]) =\< 3.0 X institutional upper limit of normal (ULN) * Creatinine within normal institutional limits OR * Creatinine clearance \>= 60 mL/min/1.73 m2 for patients with creatinine levels above institutional normal * If the patient is a cancer survivor, all of the following criteria must be met * Patient has undergone potentially curative therapy from all prior malignancies * Patient must have no evidence of any prior malignancies for at least 5 years with no evidence of recurrence (except for successfully treated cervical carcinoma in situ, lobular carcinoma in situ of the breast, contralateral ductal carcinoma in situ (DCIS) treated with mastectomy or lumpectomy and radiation but without tamoxifen treatment, or non-melanoma skin cancer with no evidence of recurrence) * Patient must be deemed by her treating physician to be at low risk (\< 30%) for recurrence from prior malignancies * Patient must be able to understand and willing to sign a written informed consent document Exclusion Criteria: * Patient must not have inflammatory breast cancer defined as clinically significant erythema of the breast and/or documented dermal lymphatic invasion (not direct skin invasion by tumor or peau d'orange without erythema) * Patient must not have had prior treatment for invasive breast cancer, including radiation, endocrine therapy, chemotherapy, or investigational agent; patients whose diagnosis was established by incision biopsy are not eligible * Patient must not have had prior DCIS in the ipsilateral breast * Patient must not have used tamoxifen for prior contralateral DCIS * Patient must not have any evidence of distant metastasis (M1) on imaging; staging scans are not mandatory but any exams performed as standard of care throughout the study period will be collected for correlation as needed * If patient does not agree to undergo mastectomy or lumpectomy after neoadjuvant therapy, she is ineligible for this study * Patient must not be receiving other investigational agents or be enrolled in another neoadjuvant clinical trial for treatment of the existing breast cancer * Pregnant and/or breastfeeding women are excluded from this study * Patient must not have any concurrent life threatening illnesses * Patient must not have undergone prior sentinel lymph node surgery; cores or FNA of lymph node are acceptable