Study of Safety, Pharmacokinetic and Efficacy of APG-5918 in Advanced Solid Tumors or Lymphomas

Inclusion Criteria: * Has an Eastern Cooperative Oncology Group (ECOG) performance status of 0 to 1 in dose escalation or 0 to 2 in dose expansion * Has a life expectancy of \>3 months * Has a malignancy: with histologically or cytologically confirmed locally advanced or metastatic solid tumors or relapsed or refractory Non-Hodgkin's Lymphoma (NHL) who have disease progression after treatment with available therapies that are known to confer clinical benefit. 1. has measurable disease based on RECIST 1.1 for advanced solid tumors including but not limited to nasopharyngeal carcinoma, castration-resistant prostate cancer, gastric cancer, ovarian clear cell carcinoma, mesothelioma, and sarcoma 2. has measurable disease based on Non-Hodgkin's Lymphoma Cheson response criteria for NHL * For subjects with B cell lymphoma: has documented EZH2 mutation status or be willing to perform EZH2 mutation status testing * For subjects with sarcoma: patients with epithelioid sarcoma or sarcoma with confirmed evidence of aberrant SMARCB1 status is preferred * For subjects with prostate cancer: patients must have evidence of castration resistance (as evidenced by confirmed elevated prostate-specific antigen (PSA) (per Prostate Cancer Working Group \[PCWG3\] criteria) and serum testosterone of castrate levels (i.e. ≤ 50 ng/dL)) * Adequate hematologic function defined as: 1. ANC ≥1.0 x 10˄9/L independent of growth factor support within 7 days of the first dose with study drug 2. Hemoglobin ≥9 g/dL without transfusion or growth factor support within 7 days of the first dose of study drug 3. Platelet count ≥ 75 x 10˄9/L without transfusion support within 7 days of the first dose of study drug * Adequate hepatic and renal function defined as: 1. AST and ALT ≤ 3 x upper limit of normal (ULN) (≤ 5 x ULN if liver metastases) 2. Calculated creatinine clearance ≥ 50 mL/min using the Cockcroft-Gault formula 3. Total Bilirubin ≤1.5 x ULN (Except if considered secondary to Gilbert's syndrome and primarily indirect bilirubinemia) * PT and aPTT ≤2 x ULN * Troponin ≤ 2 x ULN * QTcF interval ≤470ms for all genders (mean (triplicate) n =3), measured between 2-5 minutes apart * Stable brain metastases with clinically controlled neurologic symptoms * Willingness to use contraception by either true abstinence or the use of a method that is deemed effective by the investigator by both males and female patients of childbearing potential and their partners throughout the treatment period and for at least three months following the last dose of study drug. Note: Female participants of non-child-bearing potential are defined as: 1. surgically sterile, 2. postmenopausal for 12 months, or 3. receiving a stable dose of oral, implanted, transdermal or injectable contraceptive for at least 3 months with the last dose of injectable contraceptive within 2 months (Nonsurgical menopause history must be confirmed by follicle-stimulating hormone and luteinizing hormone levels as defined by established laboratory ranges) * Ability to understand and willingness to sign a written informed consent form (the consent form must be signed by the patient prior to any study-specific procedures) * Willingness and ability to comply with study procedures and follow-up examination Exclusion Criteria: * Receiving concurrent anti-cancer therapy (chemotherapy, radiation therapy, surgery, immunotherapy, hormonal therapy, targeted therapy, biologic therapy, with the exception of hormones for hypothyroidism or estrogen replacement therapy (ERT), anti-estrogen analogs, agonists required to suppress serum testosterone levels); or any investigational therapy within 14 days or 5 times of half-life of the molecule prior to the first dose of study drug * Steroid therapy for anti-neoplastic intent within 7 days prior to the first dose of the study drug * Continuance of toxicities due to prior radiotherapy, targeted therapy, immunotherapy or chemotherapeutic agents that do not recover to \< Grade 2, except alopecia or leukodermia * Has gastrointestinal conditions that could affect the absorption of APG-5918 in the opinion of the Investigator * Use of therapeutic doses of anti-coagulants is excluded, along with antiplatelet agents; low-dose anticoagulation medications that are used to maintain the patency of a central intravenous catheter are permitted * Received a biologic (G-CSF, GM-CSF, or erythropoietin) within 7 days prior to the first dose of the study drug * Failure to recover adequately, as judged by the investigator, from prior surgical procedures. Patients who have had major surgery within 28 days from study entry, and patients who have had minor surgery within 14 days of study entry. * Severe cardiac conditions defined as: 1. New York Heart Association (NYHA) class III or IV cardiac disease, including preexisting uncontrolled clinically significant arrhythmia, congestive heart failure, or cardiomyopathy 2. Unstable angina, myocardial infarction, or a coronary revascularization procedure within ≤ 3 months prior to initiation of study treatment 3. Echocardiography showing left ventricular ejection fraction (LVEF) \< 50% 4. poorly controlled hypertension, or history of poor compliance with antihypertensive drug regimens * Symptomatic brain metastases per clinical evaluation due to tumor involvement of the central nervous system (CNS). Patients with CNS tumors that have been treated are asymptomatic and who have discontinued steroids (for the treatment of CNS tumors) for ≤ 28 days may be enrolled. * Active symptomatic fungal, bacterial, and/or viral infection. Patients with well controlled human immunodeficiency virus (HIV), hepatitis B or C can be enrolled. * Prior treatment with embryonic ectoderm development (EED) inhibitors * Concurrent treatment with QT interval-prolonging drugs * Medical history of Torsades de Pointes * Patients with known or suspected allergy or hypersensitivity to drugs/compounds similar in composition to APG-5918 or other EED inhibitors * Any other condition or circumstance of that would, in the opinion of the investigator, make the patient unsuitable for participation in the study * Other malignant diseases than the ones being treated in this study with the exception of: cured malignancy without recurrence within 3 years prior to study entry; completely resected basal cell and squamous cell skin cancer; completely resected carcinoma in situ of any type * Non-Hodgkin lymphoma patients who have received prior allogeneic stem cell transplant * Severe and/or uncontrolled medical conditions that in the investigator's opinion could affect the safety of individual or impair the assessment of study result * Long-term steroid therapy, except for the following: 10 mg prednisone (or equivalent) daily or lower doses of steroids for control of nausea, vomiting, active autoimmune disease and seasonal allergies or prevention of adrenocortical insufficiency Note: topical steroids or inhaled steroids are allowed * Pregnant (confirmed by human chorionic gonadotropin (HCG) testing) or lactating women