Everolimus, Letrozole and Trastuzumab in HR- and HER2/Neu-positive Patients

Inclusion Criteria: * Signed informed consent obtained prior to any screening procedures * Performance status =\< 1 * Absolute neutrophil count (ANC) \>= 1 x 10\^9/L * Platelets \>= 75 x 10\^9/L * Hemoglobin (Hb) \> 8 g/dL * Total serum bilirubin =\< 2.0 mg/dL * Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) =\< 2.5 x upper limit of normal (ULN) (=\< 5 x ULN in patients with liver metastases) * International normalized ratio (INR) =\< 2 * Serum creatinine =\< 1.5 x ULN * Fasting serum cholesterol =\< 300 mg/dL or =\< 7.75 mmol/L and fasting triglycerides =\< 2.5 x ULN; Note: in case one or both of these thresholds are exceeded, the patient can only be included after initiation of appropriate lipid lowering medication * Histologically confirmed advanced solid tumors with HR-positivity defined as \> 1% on immunohistochemistry (estrogen receptor-positive with or without positivity for the progesterone receptor) and HER2/neu positivity (3+ on IHC and/or 2+ on IHC and FISH amplified, or by v-erb-b2 avian erythroblastic leukemia viral oncogene homolog 2 \[ERBB2\] mutation on next generation sequencing) * Must have measurable or evaluable disease * At least 4 weeks since the last dose of chemotherapy, immunotherapy, surgery, or radiation therapy (exception: patients may have received palliative low dose radiation therapy one week before treatment provided it is not given to the only targeted lesions); at least 6 weeks for therapy which is known to have delayed toxicity (nitrosoureas, mitomycin-C, and liposomal doxorubicin); at least 4 weeks (or 5 half-lives, whichever is shorter) since treatment with biologic/targeted therapies; at least 2 weeks since last hormonal therapy * Female patients must either be: post-menopausal women as defined by a) age \>= 60 years of age; b) prior bilateral oophorectomy; c) age \< 60 with at least 12 months of spontaneous amenorrhea or post-menopausal range follicle stimulating hormone (FSH) and estradiol levels or premenopausal women receiving a gonadotropin-releasing hormone agonist * Patient may have had any number of prior chemotherapy regimens in the adjuvant/neoadjuvant and/or metastatic setting (including none) * Patient may have had any number of prior treatments with anti-HER2 strategies in the adjuvant/neoadjuvant and/or metastatic setting (including none) * Patient may have had any number of prior hormonal therapies in the adjuvant/neoadjuvant and/or metastatic setting (including none) * Breast cancer patients in the expansion cohort must be hormone sensitive or have refractory disease Exclusion Criteria: * Patients currently receiving anticancer therapies or who have received anticancer therapies within 4 weeks of the start of everolimus (including chemotherapy, radiation therapy, antibody based therapy, etc.) * Known intolerance or hypersensitivity to everolimus or other rapamycin analogs (e.g. sirolimus, temsirolimus) * Known impairment of gastrointestinal (GI) function or GI disease that may significantly alter the absorption of oral everolimus * Uncontrolled diabetes mellitus as defined by glycated hemoglobin (HbA1c) \> 8% despite adequate therapy; patients with a known history of impaired fasting glucose or diabetes mellitus (DM) may be included, however blood glucose and antidiabetic treatment must be monitored closely throughout the trial and adjusted as necessary * Patients who have any severe and/or uncontrolled medical conditions such as: * Unstable angina pectoris, symptomatic congestive heart failure, myocardial infarction 6 months prior to start of everolimus, serious uncontrolled cardiac arrhythmia, or any other clinically significant cardiac disease * Symptomatic congestive heart failure of New York Heart Association class III or IV * Active (acute or chronic) or uncontrolled severe infection, liver disease such as cirrhosis, decompensated liver disease, and chronic hepatitis (i.e. quantifiable hepatitis B virus \[HBV\]-deoxyribonucleic acid \[DNA\] and/or positive hepatitis B surface antigen \[HbsAg\], quantifiable hepatitis C virus \[HCV\]-ribonucleic acid \[RNA\]) * Known severely impaired lung function (spirometry and diffusion capacity of the lungs for carbon monoxide \[DLCO\] 50% or less of normal and oxygen \[O2\] saturation 88% or less at rest on room air) * Active, bleeding diathesis * Chronic treatment with corticosteroids or other immunosuppressive agents; topical or inhaled corticosteroids are allowed * Known history of human immunodeficiency virus (HIV) seropositivity * Patients who have received live attenuated vaccines within 1 week of start of everolimus and during the study; patient should also avoid close contact with others who have received live attenuated vaccines; examples of live attenuated vaccines include intranasal influenza, measles, mumps, rubella, oral polio, bacillus Calmette-Guérin (BCG), yellow fever, varicella and TY21a typhoid vaccines * Patients who have a history of another primary malignancy, with the exceptions of: non-melanoma skin cancer, and carcinoma in situ of the cervix, uteri, or breast from which the patient has been disease free for \>= 3 years * Patients with a history of non-compliance to medical regimens or who are considered potentially unreliable or will not be able to complete the entire study * Patients who are currently part of or have participated in any clinical investigation with an investigational drug within 1 month prior to dosing * Pregnant or nursing (lactating) women * Women of child-bearing potential (WOCBP), defined as all women physiologically capable of becoming pregnant, must use highly effective methods of contraception during the study and 8 weeks after; highly effective contraception methods include combination of any two of the following: * Use of oral, injected or implanted hormonal methods of contraception or * Placement of an intrauterine device (IUD) or intrauterine system (IUS) * Barrier methods of contraception: condom or occlusive cap (diaphragm or cervical/vault caps) with spermicidal foam/gel/film/cream/ vaginal suppository * Total abstinence or * Male/female sterilization * Women are considered post-menopausal and not of child-bearing potential if they have had 12 months of natural (spontaneous) amenorrhea with an appropriate clinical profile (e.g. age appropriate, history of vasomotor symptoms) or have had surgical bilateral oophorectomy (with or without hysterectomy) or tubal ligation at least six weeks prior to randomization; in the case of oophorectomy alone, only when the reproductive status of the woman has been confirmed by follow up hormone level assessment is she considered not of child-bearing potential * Male patients whose sexual partner(s) are WOCBP who are not willing to use adequate contraception, during the study and for 8 weeks after the end of treatment * History of allergic reactions or hypersensitivity to compounds similar to trastuzumab and/or letrozole * Left ventricular ejection fraction (LVEF) \< 50% * Patients with corrected QT (QTc) interval \> 0.47 seconds * Prior exposure to more than 360 mg/m\^2 doxorubicin, more than 120 mg/m\^2 mitoxantrone, or more than 90 mg/m\^2 idarubicin, or elevated baseline cardiac troponin I * Drugs with potent cytochrome P450, family 3, subfamily A, polypeptide 4 (CYP3A4) inhibitors and inducers should be avoided during the course of treatment * Patients with active central nervous system (CNS) metastasis and/or carcinomatous meningitis; however, patients with CNS metastasis (except leptomeningeal disease) who have completed a therapy and are clinically stable for 3 weeks as defined as: (1) no evidence of new or enlarging CNS metastasis and (2) off steroids and/or anticonvulsants may be eligible * Patient is known to be hepatitis B or hepatitis C-positive (these tests are not required) * Patients with current active hepatic or biliary disease (with exception of patients with Gilbert's syndrome, asymptomatic gallstones, liver metastases or stable chronic liver disease)