Memory-like Natural Killer (NK) Cell Therapy in Patients With Renal Cell Carcinoma or Urothelial Carcinoma

Inclusion Criteria: * Histologically or cytologically confirmed, advanced or metastatic clear cell renal cell carcinoma, translocation renal cell carcinoma, chromophobe renal cell carcinoma, or urothelial carcinoma. The presence of rhabdoid or sarcomatoid differentiation is permitted if a clear cell or urothelial carcinoma component is also present. * Participants must have measurable disease, defined as at least one lesion that can be accurately measured in at least one dimension (longest diameter to be recorded for nonnodal lesions and short axis for nodal lesions) as ≥20 mm (≥2 cm) by chest x-ray or as ≥10 mm (≥1 cm) with CT scan, MRI, or calipers by clinical exam. See Section 11 (Measurement of Effect) for the evaluation of measurable disease. * Age ≥18 years. Because no dosing or adverse event data are currently available on the use of CIML NK cells in participants \<18 years of age, children are excluded from this study, but would be eligible for future pediatric trials. * Participants with clear cell RCC or UC must have progression after prior treatment failure with at least one PD-1/PD-L1 immune checkpoint inhibitor that is FDA approved for treatment of UC or RCC as of the date of informed consent. * Patients with renal cell carcinoma should also have prior treatment failure with at least one prior VEGFR TKI, or contraindication to VEGFR TKIs as determined by the treating clinician. Patients with urothelial carcinoma should have either prior treatment failure with ≥1 prior cytotoxic chemotherapy or antibody-drug conjugate. There is no limit on the number of prior lines of therapy received. * ECOG performance status ≤1 (Karnofsky ≥80%, see Appendix A). * Participants must meet the following organ and marrow function as defined below: * absolute neutrophil count ≥1,000/mcL * hemoglobin ≥8g/dL (prior transfusion permitted) * platelets ≥75,000/mcL * total bilirubin ≤ 1.5 × institutional upper limit of normal (ULN) except if secondary to Gilbert's, then \< 3 x ULN * AST(SGOT)/ALT(SGPT) ≤3.0 × institutional ULN * creatinine ≤ 2.0 OR * glomerular filtration rate (GFR) ≥40 mL/min/1.73 m2 for participants with creatinine levels above institutional ULN. * oxygen saturation ≥ 90% on room air * left ventricular ejection fraction \> 40% * No laboratory evidence of ongoing hemolysis in opinion of investigator * Participants with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, should have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, participants should be class 2B or better. * Willing to provide blood and tissue from diagnostic biopsy * Negative serum or urine pregnancy test at screening for women of childbearing potential. Highly effective contraception for female subjects of childbearing potential throughout the study if the risk of conception exists. * Ability to understand and the willingness to sign a written informed consent document. * Recipients of prior allogeneic stem cell transplantation are eligible if there is no evidence of ongoing acute or chronic graft versus host disease. Exclusion Criteria: * Participants who have had anti-tumor chemotherapy or other investigational agents within 2 weeks prior to enrollment(4 weeks for nitrosoureas or mitomycin C), or immunotherapy within 4 weeks prior. * Persisting toxicity related to prior therapy (NCI CTCAE Grade \> 1); however, alopecia, sensory neuropathy Grade ≤ 2, or other Grade ≤ 2 not constituting a safety risk based on investigator's judgment are acceptable. * Prior recipients of solid organ transplantation. * Participants who are receiving any other investigational agents. * Participants with leptomeningeal disease are excluded from this clinical trial due to their poor prognosis and because they often develop progressive neurologic dysfunction that would confound the evaluation of neurologic and other adverse events. Patients with treated brain metastases are eligible if imaging has shown stability over at least 4 weeks. * Autoimmune disease: patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn disease, are excluded from this study, as are patients with a history of active autoimmune disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[Wegener's granulomatosis\]) and motor neuropathy considered of autoimmune origin (e.g., Guillain-Barre syndrome and myasthenia gravis). Patients with diabetes type I, vitiligo, psoriasis, or hypo- or hyperthyroid diseases not requiring immunosuppressive treatment are eligible. * Clinically significant (i.e., active) cardiovascular disease: cerebral vascular accident/stroke (\< 6 months prior to enrollment), myocardial infarction (\< 6 months prior to enrollment), unstable angina, congestive heart failure (≥ New York Heart Association Classification Class II), or unstable cardiac arrhythmia requiring medication. Patients with rate controlled atrial fibrillation / atrial flutter on stable medical therapy are eligible. * Other uncontrolled intercurrent illness including, but not limited to, ongoing or active infection requiring systemic therapy, or uncontrolled psychiatric conditions including recent (within the past year) or active suicidal ideation or behavior, or other psychiatric illness/social situations that would limit compliance with study requirements, or laboratory abnormalities that may increase the risk associated with study participation or study treatment administration or may interfere with the interpretation of study results and, in the judgment of the investigator, would make the patient inappropriate for entry into this study. * Participants with known additional malignancy that is progressing or requires active treatment, or history of other malignancy within 2 years of enrollment, with the exception of cured basal cell or squamous cell carcinoma of the skin, superficial bladder cancer, prostate intraepithelial neoplasm, carcinoma in situ of the cervix, or other non-invasive or indolent malignancy, or cancers from which the patient has been disease-free for \> 1 year after treatment with curative intent. * No systemic corticosteroid therapy (≥ 10 mg of prednisone or equivalent dose of systemic steroids for non-autoimmune indications for at least 2 weeks prior to enrollment). * Pregnant women are excluded from this study because of the unknown teratogenic risk of CIML NK cells and N-803 and with the potential for teratogenic or abortifacient effects by fludarabine/cyclophosphamide chemotherapy regimen. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with CIML NK cells and N-803, breastfeeding should be discontinued if the mother is treated on this study. For this reason, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study or 12 months after last treatment, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 12 months after completion of the last CIML NK cell and/or last N-803 administration. * Participants with history of positive HIV testing are ineligible because of the potential for pharmacokinetic interactions with anti-retroviral agents and the treatments used in this study. In addition, these participants are at increased risk of lethal infections when treated with marrow-suppressive therapy. * Individuals with Hepatitis B virus (HBV) or hepatitis C virus (HCV) infection at screening (positive HBV surface antigen or HCV RNA if anti-HCV antibody screening test positive) are ineligible as they are at increased risk of lethal treatment-related hepatotoxicity. * History of allergic reactions attributed to compounds of similar chemical or biologic composition to N-803 or other agents used in study. * Receipt of a live vaccine within 2 weeks prior to enrollment. * Known prior severe allergic/anaphylactic reactions to murine-based antibody therapy or iron dextran, as the CIML NK cell product contains similar reagents at end of manufacturing/infusion. * Prior history of Grade 2 or higher hemolytic anemia (\>/= 2g decrease in hemoglobin plus laboratory evidence of hemolysis) from any cause.