A Phase 1 Study of Alisertib Participants With Advanced Solid Tumors Including Castration-Resistant Prostate Cancer Receiving a Standard Docetaxel Regimen

Inclusion Criteria: Each participant must meet all of the following inclusion criteria to be enrolled in the study: * 18 years or older * Histologically or cytologically confirmed advanced tumors and candidates for docetaxel treatment * Measurable or evaluable disease is required. Participants must have clinical evidence of progressive disease or persistent disease * Participants with castration-resistant prostate cancer (CRPC) are required to have * Pathologically confirmed adenocarcinoma of the prostate * Evidence of metastatic disease on bone scan or other imaging. Participants with prostate-specific antigen (PSA) elevation as the only manifestation of disease are not eligible. * Progressive disease after at least 1 hormonal treatment with documented testosterone levels less than 50 ng/dl * Concurrent use of an agent for testosterone suppression (e.g., luteinizing hormone-releasing hormone \[LHRH\] agonist) is required if the participants has not been surgically castrated * Eastern Cooperative Oncology Group (ECOG) performance status 0-2 * Recovered to less than or equal to Grade 1 toxicity (CTCAE), to participant's baseline status (except alopecia) or deemed irreversible from the effects of prior cancer therapy and must have evidence of progressive or persistent disease * Adequate bone marrow, liver and renal function * Any use of opiates must be stable for at least 2 weeks prior to study entry * Female participants who are postmenopausal for at least 1 year OR are surgically sterile OR if of childbearing potential, agree to practice 2 effective methods of contraception at the same time * Male participants who agree to practice effective barrier contraception during the entire study and through 6 months after the last dose of study drug OR agree to abstain from heterosexual intercourse * Voluntary written consent * Willing to comply with scheduled visits, treatment plan, laboratory tests and other trial procedures * Suitable venous access for blood sampling Exclusion Criteria: Participants meeting any of the following exclusion criteria are not to be enrolled in the study: * Female participants who are lactating or pregnant * Antineoplastic therapy or any experimental therapy within 21 days before the first dose of alisertib * Prior or current investigational therapies within 4 weeks before the first dose of MLN8237 * Concurrent investigational treatment of treatment with any investigational products within 28 days before the first dose of alisertib * Radiotherapy to greater than 40% of bone marrow or any radiotherapy (except localized, small field radiation) within 4 weeks prior to enrollment, unless reviewed and approved by the medical monitor * Nitrosoureas or mitomycin-C within 6 weeks before the first dose of alisertib. * Autologous stem cell transplant within 3 months before the first dose of alisetib, or prior allogeneic stem cell transplant at any time. * Use of enzyme-inducing antiepileptic drugs such as phenytoin, carbamazepine or phenobarbital, or rifampin, rifabutin, rifapentine or St. John's wort within 14 days prior to the first dose of alisertib * For CRPC participants: * Radiotherapy or antiandrogen therapy for prostate cancer within 4 weeks prior to enrollment * Prior treatment with antineoplastic chemotherapy or radioisotopes for advanced prostate cancer * Use of products known to affect PSA levels within 4 weeks of enrollment * Major surgery within 4 weeks of study enrollment * Uncontrolled high blood pressure * Participants with abnormal gastric or bowel function or who require continuous treatment with antacids or proton pump inhibitors * Participants receiving chronic steroid therapy other than the following: low dose steroid for the control of nausea and vomiting, topical steroid, inhaled steroid or use of dexamethasone * Known severe hypersensitivity to docetaxel or other drugs formulated in polysorbate 80 * Comorbid condition or unresolved toxicity that would preclude administration of docetaxel * Prior history of Grade 2 or greater neurotoxicity or any toxicity that has not resolved to Grade 1 or below * Symptomatic brain or other CNS metastasis * Diagnosis or treatment of another malignancy within 2 years preceding first dose of study drug except nonmelanoma skin cancer or in situ malignancy completely resected * Known history of human immunodeficiency virus (HIV) infection, hepatitis B, or hepatitis C * Participants requiring full systemic anticoagulation * Prior allogeneic bone marrow or other organ transplant * Active infection requiring systemic therapy within 14 days preceding first dose, or other serious infection * History of hemorrhagic or thrombotic cerebrovascular event in the past 12 months * Serious medical or psychiatric illness that could interfere with protocol completion * Inability to swallow oral medication * Prior treatment with more than 3 myelosuppressive cytotoxic chemotherapy regimens * Prior treatment with more than 1 prior taxane-containing regimen