A Study of Chemoimmunotherapy for the Treatment of Men With Neuroendocrine or Aggressive Variant Metastatic Prostate Cancer

Inclusion Criteria: 1. Neuroendocrine-like prostate cancer, based on histology OR based on clinical presentation as defined by meeting one of the two below criteria. All subjects must submit their primary tumor or metastatic biopsy pathology specimens to the Duke Cancer Institute where they will be centrally reviewed by Duke Pathology. Central Duke pathologic review is not required for screening but rather for confirmation of histologic subtype. Local pathologic review is sufficient for eligibility determination. 1. Criterion 1: Presence of 1 of 3 histologically proven diagnoses: 1) Primary small cell carcinoma of the prostate, defined by classic histologic features such as small tumor cells with scanty cytoplasm, darkly stained nuclei with homogeneous chromatin pattern. The tumor cells do not form glandular structure but grow as solid sheets with frequent mitotic figures and necrosis; 2) Intermediate atypical carcinoma of the prostate, which has histologic features distinct from small cell carcinoma or adenocarcinoma. The tumor grows as solid sheets or vague glandular structures. The tumor cells have moderate amounts of cytoplasm and centrally located, round and regular nuclei with fine, granular and homogeneous chromatin. Mitosis and necrosis are absent; 3) mixed histology tumors of the prostate, containing both adenocarcinoma and neuroendocrine or small cell components. 2. Criterion 2: Presence of histologically proven adenocarcinoma of the prostate without any sign of neuroendocrine or small cell histology that is radiographically progressing with the following poor risk features: i. Prior progression despite therapy with abiraterone acetate, darolutamide or apalutamide and/or enzalutamide. ii. At least one of the following: 1) Visceral metastases; 2) Low PSA (\<10 ng/mL) with either A. bulky lymphadenopathy or pelvic mass (\>5 cm) or B. high volume (\>20) bone metastases; 3) Short interval (\<6mo) to CRPC following initiation of hormonal therapy 4) Pathogenic alterations in two of three genes: TP53, RB1, and PTEN. 5) Predominantly lytic bone metastases on imaging, 6) Presence of neuroendocrine markers on histology (positive staining of chromogranin A or synaptophysin) or in serum (abnormal high serum levels for chromogranin A or gastrin releasing peptide (GRP)) at initial diagnosis or at progression; 7) Any of the following in the absence of other causes: A. elevated serum LDH (\>= IULN); B. malignant hypercalcemia; C. elevated serum CEA (\>2x IULN). 2. Available archival tumor tissue for pathologic review and correlative studies. Tumor tissue (localized or metastatic) does not need to be received but rather identified and available (slides and/or blocks) to be sent to Duke. 3. Documented progressive metastatic CRPC as determined by the provider based on at least one of the following criteria: 1. PSA progression defined as 25% increase over baseline value with an increase in the absolute value of at least 2.0 ng/mL that is confirmed by another PSA level with a minimum of a 1 week interval and a minimum PSA of 2.0 ng/mL. Note: If confirmed rise is the only indication of progression, a minimal starting value of 1.0 ng/mL is acceptable, unless pure small-cell carcinoma. 2. Soft-tissue progression based on new lesions or growth of existing soft tissue metastases. 3. Progression of bone metastasis with one or more new bone lesion(s) by imaging. 4. Castrate levels of serum total testosterone (\<50 ng/dl) OR ongoing documented ADT. a. These criteria are not required when pure small cell prostate cancer is present. 5. Karnofsky performance status of 70 or higher. 6. Acceptable initial laboratory values within 14 days of Cycle 1 Day 1 7. Age \>18 8. Subjects with a partner who is a woman of child-bearing potential must agree to use one form of highly effective contraception as detailed in Section 8.3 of this protocol during the treatment period with cabazitaxel. Subjects receiving cabazitaxel or nivolumab must also refrain from donating sperm during this period. 9. Willing and able to provide written informed consent and HIPAA authorization for the release of personal health information. 10. Life expectancy of over 3 months as determined by treating physician. Exclusion Criteria: 1. Has received prior therapy for prostate cancer with abiraterone or androgen receptor antagonists (e.g. enzalutamide darolutamide, apalutamide) within two weeks of study treatment initiation. 2. Has received prior therapy with an anti-PD-1, anti-PD-L1, or anti PD L2 agent or with an agent directed to another stimulatory or co-inhibitory T-cell receptor (eg, CTLA-4, OX 40, CD137). 3. Has received other prior systemic anti-cancer therapy not otherwise addressed by other eligibility criteria including investigational agents within 4 weeks prior to study treatment initiation 4. Prior receipt of cabazitaxel chemotherapy or 2 or more chemotherapy regimens in the mCRPC setting. 5. Has received prior radiotherapy within 2 weeks of start of study intervention. Participants must have recovered from all radiation-related toxicities, not require corticosteroids, and not have had radiation pneumonitis. A 1-week washout is permitted for palliative radiation (≤2 weeks of radiotherapy) to non-CNS disease. 6. Has received a live vaccine within 30 days prior to the first dose of study drug. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, varicella/zoster (chicken pox), yellow fever, rabies, Bacillus Calmette-Guérin (BCG), and typhoid vaccine. Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (eg, FluMist®) are live attenuated vaccines and are not allowed. 7. Is currently participating in or has participated in a study of an investigational agent or has used an investigational device within 4 weeks prior to the first dose of study intervention. 8. Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study drug. 9. Has a history of a second malignancy, unless potentially curative treatment has been completed with no evidence of malignancy for 2 years. 10. Has known active untreated CNS metastases and/or carcinomatous meningitis. Participants with previously treated brain metastases may participate provided they are radiologically stable, i.e. without evidence of progression for at least 4 weeks by repeat imaging (note that the repeat imaging should be performed during study screening), clinically stable and without requirement of steroid treatment greater than prednisone 10mg (or equivalent) for at least 14 days prior to first dose of study intervention. 11. Has active autoimmune disease that has required systemic treatment in the past 2 years (i.e. with use of disease modifying agents, corticosteroids or immunosuppressive drugs). Replacement therapy (eg., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment and is allowed. 12. Has a history of (non-infectious) pneumonitis that required steroids or has current pneumonitis. 13. Has an active infection requiring systemic therapy. 14. Has a known uncontrolled Human Immunodeficiency Virus (HIV) infection based on detectable HIV viral load and abnormal CD4 count of \<350/mm3. 15. Has a known active Hepatitis B (defined as Hepatitis B surface antigen \[HBsAg\] reactive) or known active Hepatitis C virus (defined as HCV RNA \[qualitative\] is detected) infection. 16. Has a known active TB (Bacillus Tuberculosis) infection. 17. Has ≥ Grade 2 neuropathy. 18. Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the study, interfere with the subject's participation for the full duration of the study, or is not in the best interest of the subject to participate, in the opinion of the treating investigator. 19. Has known current psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. 20. Has had an allogenic tissue/solid organ transplant.