Genetically Engineered Cells (MAGE-A1-specific T Cell Receptor-transduced Autologous T-cells) and Atezolizumab for the Treatment of Metastatic Triple Negative Breast Cancer, Urothelial Cancer, or Non-small Cell Lung Cancer

Inclusion Criteria: * Tissue confirmation of triple negative breast cancer (TNBC), urothelial carcinoma or non-small cell lung cancer (NSCLC) and expression of MAGEA1: Participants must have metastatic disease. Confirmation of diagnosis must be or have been performed by internal pathology review of archival, initial or subsequent biopsy or other pathologic material at Fred Hutchinson Cancer Research Center (FHCRC)/Seattle Cancer Care Alliance (SCCA)/University of Washington Medical Center (UWMC). Patients with TNBC must meet the American Society of Clinical Oncology - College of American Pathologists (ASCO-CAP) definition of negative estrogen, progesterone and HER2 receptor expression. Baseline tissue will be stained to confirm MAGE-A1 expression * Measurable disease by RECIST 1.1 criteria: Participants must have measurable disease, defined as at least one target lesion that can be measured in at least one dimension (longest diameter to be recorded) as \>= 10 mm, unless lymph node in which case short axis must be \>= 15 mm. Baseline imaging (for example diagnostic computed tomography \[CT\] chest/abdomen/pelvis and imaging of the affected extremity as appropriate), brain imaging (magnetic resonance imaging \[MRI\] or CT scan) must be obtained within 45 days of prior to start of first planned FH-MAGEA1-A2TCR infusion. MRI can be substituted for CT in patients unable to have CT contrast. Measurable disease is not required for purposes of leukapheresis /cell manufacturing storage for patients who meet HLA and expression criteria but not standard treatment criteria * Previous treatment with standard of care (SOC) Food and Drug Administration (FDA)-approved therapies. Patients with NSCLC who have actionable somatic mutations or alterations in EGFR, ROS1 and ALK with FDA-approved drug therapy options will be eligible for study only after treatment with targeted therapies for those mutations have been offered or received. Patients with urothelial carcinoma who are candidates for enfortumab vedotin (enfortumab vedotin-ejfv) will be eligible for study after prior treatment with enfortumab vedotin-ejfv has been offered or received \* Note: Participants will be eligible to enroll before standard therapy is received in order to expedite leukapheresis/cell manufacturing as long as the aforementioned agent is administered prior to lymphodepletion and/or cell infusion * Previous treatment with PD-1 axis inhibitor: Patients in Phase I/2 must have been offered or been previously treated with at least one dose of a PD-L1 axis inhibitor (e.g. PD-1 or PD-L1 inhibiting monoclonal antibody such as pembrolizumab, nivolumab, avelumab, atezolizumab, durvalumab). If received, they must have either developed progression or still have detectable disease and not have developed Common Terminology Criteria for Adverse Events (CTCAE) grade 3 or higher toxicity while on treatment. Patients may have received 1 or more prior systemic regimens for metastatic TNBC or NSCLC. There is no upper limit on prior regimens. Patients may have received prior anti-PD-1/anti-PD-L1 in the neoadjuvant or adjuvant setting * HLA type HLA-A\*02:01: Participants must be HLA-A\*02:01 in order for infused transgenic T cells in order to insure recognition of antigen-MHC complexes. HLA typing should be determined though a molecular approach in a clinical laboratory licensed for HLA typing * Life expectancy must be anticipated to be \> 3 months at trial entry * 18 years or older * Capable of understanding and providing a written informed consent * If fertile, willingness to comply with reproductive requirements * Eastern Cooperative Oncology Group (ECOG) performance status of 0, 1, or 2 * Tumor tissue amenable to safe biopsy and subject willing to undergo serial tumor biopsies: Should there be no tumor tissue that is accessible for biopsy, patients will still be considered for participation, at discretion of the sponsor and in consultation with the investigator. Similarly, should an investigator determine that a biopsy cannot be performed safely for clinical reasons biopsies may be cancelled or retimed after confirming plan with the sponsor * Participants must be at least three weeks from last systemic treatment at the time of cell collection: At least 3 weeks must have passed since any: immunotherapy (for example, T-cell infusions, immunomodulatory agents, interleukins, vaccines), small molecule or chemotherapy cancer treatment, other investigational agents. There is no washout period for radiation, so long as radiated lesion is not the lesion being evaluated for RECIST measurements on the protocol. Bisphosphonates are permitted but the concurrent treatment with RANK ligand inhibitors (i.e., denosumab) is not permitted within 8 weeks of treatment * Serum creatine \< 2.5 mg/dL or estimated glomerular filtration rate (eGFR) \> 30 mL/min * Total bilirubin (tBili) \< 3.0 mg/dL. Patients with suspected Gilbert syndrome may be included if Tbili \> 3 but no other evidence of hepatic dysfunction * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) \< 5 x upper limit of normal (ULN) * =\< grade 1 dyspnea and oxygen saturation (SaO2) \>= 92% on ambient air. If pulmonary function tests (PFTs) are performed based on the clinical judgement of the treating physician, patients with forced expiratory volume in 1 second (FEVI) \>= 50% of predicted and carbon monoxide diffusing capability test (DLCO) (corrected) of \>= 40% of predicted will be eligible * Patients 60 years of age or older are required to have left ventricular ejection fraction (LVEF) evaluation performed within 60 days prior to enrollment. LVEF may be established with echocardiogram or multigated acquisition scan (MUGA) scan, and left ejection fraction must be \>= 35%. Cardiac evaluation for other patients is at the discretion of the treating physician * Absolute neutrophil count (ANC) \> 500 cells/ mm\^3 Exclusion Criteria: * Expression of HLA B\*4901: participants will be excluded due to the risk of alloreactivity * Participants of childbearing potential must have a negative serum pregnancy test within 14 days prior to enrollment. Childbearing potential is defined as women who have not been surgically sterilized and who are not postmenopausal (free of menses for at least 1 year) * Patients with active autoimmune disease requiring immunosuppressive therapy are excluded. Case-by-case exemptions are possible with approval by principal investigator (PI) * Prior solid organ transplant or allogenic hematopoietic stem cell transplant: Kidney transplant patients will be considered on a case-by-case basis requiring discussion with PI. If kidney transplant, patient must have dialysis access, dialysis plan, supportive nephrologist, willingness to stop transplant immunosuppression, and express understanding that rejection is possible outcome. Dialysis or costs related to transplant kidney will not be supported by the study. Participants having had any other solid organ transplants will be excluded, as will those with any history of allogeneic stem cell transplant * Corticosteroid therapy at a dose equivalent of \> 0.5 mg/kg of prednisone-equivalent per day * Concurrent use of other investigational anti-cancer agents * Chronic lymphocytic leukemia (CLL) or other active hematologic malignancy * Active uncontrolled infection: Human immunodeficiency virus (HIV) positive participants on highly active anti-retroviral therapy (HAART) with a CD4 count \> 500 cells/mm\^3 are considered controlled, as are individuals with a history of hepatitis C who have successfully completed antiviral therapy with an undetectable viral load, and those with hepatitis B who have, per standard practice, hepatitis well-controlled on medication (e.g., AST and ALT \< 5 x ULN) * Participants may not have uncontrolled or concurrent illness including, but not limited to, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Participants with small asymptomatic brain metastases (\< 1 cm) or those with brain metastases previously treated with surgery or radiotherapy will be considered for inclusion at discretion of principal investigator, so long as other eligibility criteria are met. * For patients in phase 1/2, grade 3 or higher immune-mediated toxicity to any prior PD-L1 axis blocking agent * Active treatment for prior immune related adverse event to any immunotherapy: Participants receiving ongoing treatment for prior serious immune related adverse events are excluded, with exception of hormone supplementation or corticosteroid therapy at equivalent of up to 0.5 mg/kg prednisone per day, unless otherwise approved by PI * Study participants must not have significant active underlying neurologic disease, unless approved by PI. Neuropathy related to diabetes or prior chemotherapy is acceptable * Other medical, social, or psychiatric factor that interferes with medical appropriateness and/or ability to comply with study, as determined by the PI