Avelumab or Hydroxychloroquine With or Without Palbociclib to Eliminate Dormant Breast Cancer

Inclusion Criteria: * Bone marrow aspirate after completion of all definitive therapy demonstrates detectable DTCs (via IHC) as performed by central laboratory assessment at University of Pennsylvania. * History of stage II-III histologically-confirmed ER+/Her2 neg invasive breast cancer with no evidence of recurrent local or distant disease (by American Joint Committee on Cancer 7th edition). Patients with bilateral breast cancer are eligible, so long as both cancers are ER+/Her2 neg, at least one meets other eligibility criteria and patient is treated with curative intent. For patients who undergo neoadjuvant therapy, eligibility is based upon pathologic stage of residual disease at surgery. * ER+/Her2 neg receptor status on breast primary tumor (by American Society of Clinical Oncology/College of American Pathologists guidelines). Any partial response (PR) status is allowed. Tumors that are ER negative and PR positive are not eligible. Patients who undergo neoadjuvant therapy are eligible if either the pre-treatment biopsy or residual disease at surgery is ER+/Her2 neg. * Patients must have completed all primary and adjuvant therapy (including surgery, chemotherapy, and radiation) with the exception of adjuvant endocrine therapy. Prior treatment-related toxicity must be resolved to ≤ Grade 1 with the exception of alopecia and peripheral neuropathy, prior to study enrollment. * Patients may have received prior CDK4/6 inhibitor therapy with an agent other than Palbociclib. Patients must have discontinued CDK4/6 inhibitor at least 6 months prior to screening. * Patients must be receiving adjuvant endocrine therapy at the time of enrollment. Patients are eligible to enroll within 2-7 years after initiation of adjuvant endocrine therapy. Use of tamoxifen as adjuvant endocrine therapy during study treatment is not allowed on hydroxychloroquine arms due to the potential drug-drug interaction with hydroxychloroquine. However, patients on tamoxifen at the time of screening may enroll on the treatment trial if switched to an aromatase inhibitor at least 21 days prior to starting study therapy in the event patient is randomized to a hydroxychloroquine containing arm. Premenopausal patients on concurrent ovarian suppression are eligible. Patients on any other adjuvant endocrine therapy, including any investigational therapy, are ineligible. * Patients receiving bone modifying agents (bisphosphonates or rank-ligand inhibitors) at the time of screening may continue this therapy. Bone modifying agents may not be initiated while receiving study treatment. * No concurrent enrollment on another investigational therapy clinical trial. * Men and women, age ≥ 18 years. * No contraindications to the study medications (refer to Section 7.2) or uncontrolled medical illness. * Adequate bone marrow, liver, and renal function and other parameters. * Ability to speak and understand English Exclusion Criteria: * Patients with a history of another prior invasive breast cancer are ineligible. Patients with prior Ductal carcinoma in situ (DCIS) of the breast are eligible if this was diagnosed \> 5 years prior to enrollment. Patients with prior invasive malignancy other than breast cancer are eligible if they have been disease-free for at least 5 years prior to enrollment. * Patients receiving chronic, high dose systemic treatment with corticosteroids defined as: chronic use of cortisone \>50mg; hydrocortisone \>40mg, prednisone \>10mg, methylprednisone \>8mg or dexamethasone \>1.5mg; or another immunosuppressive agent. Topical or inhaled corticosteroids are allowed. * EKG demonstrating QT interval corrected (QTC) \> 480 ms * Any severe and/or uncontrolled medical conditions or other conditions that could affect subject participation in the study including: * Chronic autoimmune disease * History or evidence of increased cardiovascular risk including any of the following: * Current clinical significant uncontrolled arrhythmias. Exception: Subjects with controlled atrial fibrillation * History of acute coronary syndromes (including myocardial infarction and unstable angina), coronary angioplasty, or stenting within 6 months prior to enrollment * Current ≥ Class II congestive heart failure as defined by New York Heart Association * History of pneumonitis/interstitial lung disease or severely impaired lung function with a previously documented spirometry and Diffusing Capacity of Lung for Carbon Monoxide (DLCO) that is 50% of the normal predicted value (these tests not required at screening; prior results, if performed for standard of care should be referenced) and/or O2 saturation that is 88% or less at rest on room air * Uncontrolled diabetes * Active (acute or chronic) or uncontrolled severe infections * Liver disease such as cirrhosis, chronic active hepatitis or chronic persistent hepatitis * HIV positive patient who are receiving combination anti-retroviral therapy are ineligible because of the potential for pharmacokinetic interactions or increased immunosuppression with Palbociclib. However, HIV per se is not a contraindication to study participation and HIV testing is not required. * Impairment of gastrointestinal function or gastrointestinal disease that may significantly alter the absorption of hydroxychloroquine (e.g., ulcerative disease, uncontrolled nausea, vomiting, diarrhea, malabsorption syndrome or small bowel resection) * Patients with an active, bleeding diathesis. Patients receiving therapeutic anticoagulation are not eligible for study participation. * History of retinopathy or retinal vein occlusion * Female patients who are pregnant or breast feeding, or adults of reproductive potential who are not using effective birth control methods.