Abiraterone Acetate and Prednisone With or Without Veliparib in Treating Patients With Metastatic Castration-Resistant Prostate Cancer

Inclusion Criteria: * Have a histologic or cytologic diagnosis of prostate cancer * Have progressive metastatic castration-resistant prostate cancer, on androgen-deprivation therapy, based on at least one of the following criteria: * PSA progression defined as 25% increase over baseline value with an increase in the absolute value of at least 2 ng/mL that is confirmed by another PSA level with a minimum of a 1-week interval with a minimum PSA of 2 ng/mL * Progression of bidimensionally measurable soft tissue (nodal metastasis) assessed within one month prior to registration by a computed tomography (CT) scan or magnetic resonance imaging (MRI) of the abdomen and pelvis * Progression of bone disease (evaluable disease) (new bone lesion\[s\]) by bone scan * Agree to undergo a biopsy of at least 1 metastatic site for gene-fusion status analysis; adequate archival metastatic tissue can be used if available in lieu of a biopsy; patients will only be eligible for protocol therapy if the biopsy has tumor and the tissue is evaluable for ETS fusion status * Eastern Cooperative Oncology Group (ECOG) performance status of 0-2 * Have testosterone \< 50 ng/dL; patients must continue primary androgen-deprivation with a luteinizing hormone-releasing hormone (LHRH) analogue if they have not undergone orchiectomy * Patients must discontinue antiandrogen therapy for at least 4 weeks (e.g. flutamide, bicalutamide, nilutamide) prior to registration with no evidence of a falling PSA after washout; patients on steroids are eligible as long as they will be switched to prednisone * Have no prior exposure to cytochrome 450 family 17(CYP-17) (other than ketoconazole) or PARP inhibitors for prostate cancer; patients with prior exposure to ketoconazole are eligible * Patients with up to 2 prior chemotherapy regimens are eligible * Obtained within 14 days prior to registration: White blood cells (WBC) \>= 3,000/ul * Obtained within 14 days prior to registration: Absolute neutrophil count (ANC) \>= 1,500/ul * Obtained within 14 days prior to registration: Platelet count \>= 100,000/ul * Obtained within 14 days prior to registration: Serum creatinine =\< 1.5 x the institutional upper limits of normal or corrected creatinine clearance of \>= 50 mg/ml/hr/1.73 m\^2 body surface area (BSA) * Obtained within 14 days prior to registration: Potassium \>= 3.5 mmol/L * Obtained within 14 days prior to registration: Bilirubin within the institutional limits of normal * Obtained within 14 days prior to registration: Serum glutamic oxaloacetic transaminase (SGOT) (aspartate aminotransferase \[AST\]) =\< 2 times upper limit of normal * Obtained within 14 days prior to registration: Serum glutamate pyruvate transaminase (SGPT) (alanine aminotransferase \[ALT\]) =\< 2 times upper limit of normal * Measured within 28 days prior to administration of ABT-888: \>= 10.0 g/dL hemoglobin (Hb) with no blood transfusion in the past 28 days * Men must agree to use effective contraception during treatment and for at least 1 week after the last administration of therapy * Patients must be able to take oral medication without crushing, dissolving, or chewing tablets * Patients may have received prior radiation therapy or surgery; however, at least 21 days must have elapsed since completion of radiation therapy or surgery and patient must have recovered from all side effects at the time of registration * Ability to understand and the willingness to sign a written informed-consent document that is approved by the local institutional review board Exclusion Criteria: * Patients may not be receiving any other investigational agents; any prior investigational products must be stopped at least 14 days (2-week washout) prior to registration * Patients who have had chemotherapy, radiotherapy or oral antifungal agents (ketoconazole, itraconazole, fluconazole) within 3 weeks prior to entering the study or those who have not recovered (e.g. back to baseline or grade 1) from adverse events due to agents administered more than 3 weeks earlier * There is a potential drug interaction when abiraterone is concomitantly used with a cytochrome P450 family 2, subfamily D, polypeptide 6 (CYP2D6) substrate narrow therapeutic index (e.g., thioridazine, dextromethorphan), or strong cytochrome P450 family 3, subfamily A, polypeptide 4 \[CYP3A4\] inhibitors (e.g., atazanavir, erythromycin, indinavir, itraconazole, Ketoconazole, nefazodone, nelfinavir, ritonavir, saquinavir, telithromycin, and voriconazole) or strong inducers (e.g., carbamazepine, phenobarbital, phenytoin, rifabutin, rifampin, rifapentine); caution should be used when patients are on one of these drugs * Patients with history of active seizures are not eligible * Patients with a history of pituitary or adrenal dysfunction, active or symptomatic viral hepatitis, or chronic liver disease are not eligible * Patients with known brain metastases should be excluded from this clinical trial * History of allergic reactions attributed to compounds of similar chemical or biologic composition to ABT-888 or abiraterone * Patients may continue on a daily multi-vitamin, calcium and vitamin D, but all other herbal, alternative and food supplements (i.e. PC-Spes, saw palmetto, St John's wort, etc.) must be discontinued before registration; patients must not be planning to receive any concurrent cytotoxic chemotherapy, surgery, or radiation therapy during protocol treatment; hormonal-acting agents (including diethylstilbestrol/DES, aldosterone, and spironolactone) are forbidden during the trial and must be stopped prior to registration; no washout period will be required for any of these agents; patients on megestrol acetate for hot flashes are allowed to continue therapy * Patients on stable doses of bisphosphonates or denosumab which have been started prior to registration may continue on this medication, patients who are not on bisphosphonates or denosumab are eligible as long as they initiate therapy prior to registration * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure (New York Heart Association class III and IV heart failure), unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements or concurrent medications that alter cardiac conduction * Patients with a "currently active" second malignancy other than non-melanoma skin cancers are not eligible; patients are not considered to have a "currently active" malignancy if they have completed all therapy and are now considered without evidence of disease for 1 year * Human immunodeficiency virus (HIV)-positive patients on combination antiretroviral therapy are ineligible * Patients with treatment-related acute myeloid leukemia (AML) (t-AML)/myelodysplastic syndrome (MDS) or with features suggestive of AML/MDS. Prior allogeneic bone marrow transplant or double umbilical cord blood transplantation