Neo-adjuvant Therapy With Anastrozole Plus Pazopanib in Stage II and III ER+ Breast Cancer

Inclusion Criteria: * Subjects must provide written informed consent prior to performance of study-specific procedures or assessments, and must be willing to comply with treatment and follow-up; procedures conducted as part of the subject's routine clinical management (e.g., blood count, imaging study) and obtained prior to signing of informed consent may be utilized for screening or baseline purposes provided these procedures are within the protocol determined screening or baseline timeframes, and equivalent to the specifications in the protocol; also note, a written waiver/approval from the Investigator/Sponsor is still required * Histologically confirmed diagnosis of estrogen receptor-positive (ER+), HER2 negative breast cancer, with clinical stage II or III disease * Stage IIA T0-1 N1 M0, T2 N0 M0, OR * Stage IIB T2 N1 M0, T3 N0 M0 OR * Stage IIIA T0-2 N2 M0, T3 N1-2 M0, OR * Stage IIIB T4 N0-2 M0 * Eastern Cooperative Oncology Group (ECOG) performance status of 0-2 * No evidence of distant metastatic disease * Baseline Oncotype DX score of 29 or lower; patients with known Oncotype DX recurrence score (RS) of 30 or greater are not eligible * Absolute neutrophil count (ANC) \>= 1.5 x 10\^9/L * Hemoglobina \>= 10 g/dL (5.6 mmol/L); subjects may not have had a transfusion within 7 days of screening assessment * Platelets \>= 100 x 10\^9/L * Prothrombin time (PT) or international normalized ratio (INR) =\< 1.2 x upper limit of normal (ULN); subjects receiving anticoagulant therapy are eligible if their INR is stable and within the recommended range for the desired level of anticoagulation * Activated partial thromboplastin time (aPTT) =\< 1.2 x ULN * Total bilirubin =\< 1.5 x ULN * Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) =\< 2.5 x ULN; concomitant elevations in bilirubin and AST/ALT above 1.0 x ULN is not permitted * Serum creatinine =\< 1.5 mg/dL (133 umol/L) or, if \> 1.5 mg/dL: calculated creatinine clearance (CLCR) \>= 50 mL/min * Urine protein to creatinine ratio (UPC) \< 1; if UPC \>= 1, then a 24-hour urine protein must be assessed; subjects must have a 24-hour urine protein value \< 1 g to be eligible * A female is eligible to enter and participate in this study if she is: * Is post-menopausal * Subjects not using hormone replacement therapy (HRT) must have experienced total cessation of menses for \>= 1 year and be greater than 45 years in age, OR, in questionable cases, have follicle stimulating hormone (FSH) value \> 40 mIU/mL and an estradiol value \< 40 pg/mL (\< 140 pmol/L) * Subjects using HRT must have experienced total cessation of menses for \>= 1 year and be greater than 45 years of age OR have had documented evidence of menopause based on FSH and estradiol concentrations prior to initiation of HRT; patients will be required to be off of HRT for at least 2 weeks prior to initiating therapy Exclusion Criteria: * Prior malignancy; note: subjects who have had another malignancy and have been disease-free for \>= 5 years, or subjects with a history of completely resected non-melanomatous skin carcinoma or successfully treated in situ carcinoma are eligible * Known distant metastases at any site; history or clinical evidence of central nervous system (CNS) metastases or leptomeningeal carcinomatosis * Clinically significant gastrointestinal abnormalities that may increase the risk for gastrointestinal bleeding including, but not limited to: * Active peptic ulcer disease * Known intraluminal metastatic lesion/s with risk of bleeding * Inflammatory bowel disease (e.g. ulcerative colitis, Crohn's disease), or * Other gastrointestinal conditions with increased risk of perforation * History of abdominal fistula, gastrointestinal perforation, or intra-abdominal abscess within 28 days of registration to the study; clinically significant gastrointestinal abnormalities that may affect absorption of investigational product including, but not limited to: * Malabsorption syndrome * Major resection of the stomach or small bowel * Presence of uncontrolled infection * Corrected QT interval (QTc) \> 480 msecs using Bazett's formula * History of any one or more of the following cardiovascular conditions within 6 months of registration on the study: * Cardiac angioplasty or stenting * Myocardial infarction * Unstable angina * Coronary artery bypass graft surgery * Symptomatic peripheral vascular disease * Class III or IV congestive heart failure, as defined by the New York Heart Association (NYHA) * Poorly controlled hypertension \[defined as systolic blood pressure (SBP) of \>= 140 mmHg or diastolic blood pressure (DBP) of \>= 90 mm Hg\]; note: initiation of adjustment of antihypertensive medication(s) is permitted prior to study entry * History of cerebrovascular accident including transient ischemic attack (TIA), pulmonary embolism or untreated deep venous thrombosis (DVT) within the past 6 months of registration on the study; note: subjects with recent DVT who have been treated with therapeutic anti-coagulating agents for at least 6 weeks of registration to the study are eligible * Prior major surgery or trauma within 28 days of registration on the study prior to first dose of study drug and/or presence of any non-healing wound, fracture, or ulcer (procedures such as catheter placement not considered to be major) * Evidence of active bleeding or bleeding diathesis * Hemoptysis within 8 weeks of registration to the study * Any serious and/or unstable pre-existing medical, psychiatric, or other condition that could interfere with subject's safety, provision of informed consent, or compliance to study procedures * Treatment with any of the following anti-cancer therapies for the current diagnosis of stage 2-3 estrogen positive breast carcinoma: * Radiation therapy, surgery or tumor embolization within 14 days prior to first dose of pazopanib OR * Chemotherapy, immunotherapy, biologic therapy, investigational therapy or hormonal therapy within 14 days or five half-lives of a drug (whichever is longer) prior to the first dose of pazopanib * Any ongoing toxicity from prior anti-cancer therapy that is \> Grade 1 and/or that is progressing in severity, except alopecia * Concomitant anti-cancer therapies are not permitted * Patients may not be receiving any other investigational agents * History of allergic reactions attributed to compounds of similar chemical or biologic composition to Pazopanib or Anastrozole used in the study * Concomitant use of medications or substances that are inhibitors or inducers of strong CYP3A4 inhibitors are ineligible * Concomitant: the concomitant use of strong CYP3A4 inhibitors (e.g. ketoconazole, ritonavir, clarithromycin) may increase pazopanib concentrations and should be avoided; if coadministration of a strong CYP3A4 inhibitor is warranted, reduce the dose of pazopanib to 400 mg; further dose reductions may be needed if adverse effects occur during therapy; this dose is predicted to adjust the pazopanib AUC to the range observed without inhibitors; however, there are no clinical data with this dose adjustment in patients receiving strong CYP3A4 inhibitors; grapefruit juice should be avoided as it inhibits CYP3A4 activity and may also increase plasma concentrations of pazopanib * Concomitant strong CYP3A4 inducer: the concomitant use of strong CYP3A4 inducers (e.g. rifampin) may decrease pazopanib concentrations and should be avoided