JCAR014 and Durvalumab in Treating Patients With Relapsed or Refractory B-cell Non-Hodgkin Lymphoma

Inclusion Criteria: INCLUSION CRITERIA FOR SCREENING: * Relapsed or refractory diffuse large B-cell lymphoma (DLBCL), not otherwise specified (NOS); high grade B cell lymphoma with MYC and BCL2 and/or BCL6 rearrangements; primary mediastinal B-cell lymphoma (PMBCL); or DLBCL transformed from indolent histology with one of the following: * Persistent disease after first-line chemo-immunotherapy * Relapse after first-line chemo-immunotherapy and not eligible for autologous hematopoietic stem cell transplant (HCT) * Relapse or persistent disease after at least two lines of therapy or after autologous HCT * Ability to understand and provide informed consent INCLUSION CRITERIA FOR LEUKAPHERESIS AND PRE-THERAPY EVALUATION: * Screening evaluation appropriate for leukapheresis and T-cell collection * Evidence of CD19 expression on any prior or current tumor specimen or a high likelihood of CD19 expression based on disease histology INCLUSION CRITERIA FOR LYMPHODEPLETION CHEMOTHERAPY, JCAR014 AND DURVALUMAB: * Successful collection of T cells for JCAR014 manufacturing * Documentation of CD19 expression on any prior or current tumor biopsy * Internal review of histology * Detectable positron emission tomography (PET)-positive disease * Karnofsky performance status \>= 60% * Assessed by the investigator to have adequate bone marrow function to receive lymphodepleting conditioning chemotherapy * Serum creatinine \< 1.5 x age-adjusted upper limit of normal (ULN) * Alanine aminotransferase (ALT) and aspartate aminotransferase (AST) \< 3 x ULN and total bilirubin =\< 2 x ULN * Adequate pulmonary function, defined as Common Terminology Criteria for Adverse Events (CTCAE) grade =\< 1 dyspnea and oxygen saturation (SaO2) \>= 92% on room air; patients with clinically significant pulmonary dysfunction, as determined by medical history and physical exam should undergo pulmonary function testing and must have a forced expiratory volume in 1 second (FEV1) \>= 50% of predicted value or diffusing capacity of the lung for carbon monoxide (DLCO; corrected) \>= 40% of predicted value * Adequate cardiac function, defined as left ventricular ejection fraction (LVEF) \>= 35% as assessed by echocardiogram (ECHO) or multiple uptake gated acquisition (MUGA) * Women of reproductive potential (defined as all women physiologically capable of becoming pregnant) must agree to use suitable methods of contraception for 90 days after the last dose of study therapy (durvalumab or JCAR014) * Males who have partners of reproductive potential must agree to use an effective barrier contraceptive method for 90 days after the last dose of study therapy (durvalumab or JCAR014) Exclusion Criteria: EXCLUSION CRITERIA FOR SCREENING: * Subjects with known active central nervous system (CNS) involvement by malignancy; subjects with prior CNS disease that has been effectively treated will be eligible if treatment was completed at least 3 months prior to enrollment and there is no evidence of disease or stable abnormalities on repeat imaging * Planned use of corticosteroids (\> 10 mg/day prednisone or equivalent) or other systemic immunosuppression within 4 days prior to leukapheresis; topical and/or inhaled steroids are permitted * Prior treatment with any CD19 CAR T-cell therapy * Prior allogeneic HCT * Known active hepatitis B, hepatitis C, or human immunodeficiency virus (HIV) infection * Pregnant or breastfeeding women * Known exclusion criteria for leukapheresis, JCAR014, or durvalumab therapy EXCLUSION CRITERIA FOR LEUKAPHERESIS AND PRE-THERAPY EVALUATION: * Subjects with known active central nervous system (CNS) involvement by malignancy; subjects with prior CNS disease that has been effectively treated will be eligible if treatment was completed at least 3 months prior to enrollment and there is no evidence of disease or stable abnormalities on repeat imaging * Prior treatment with programmed cell death (PD)-1, PD-ligand (L)1, cytotoxic T lymphocyte-associated protein 4 (CTLA 4) targeted therapy, or tumor necrosis factor receptor superfamily (TNFRSF) agonists including CD134 (OX40), CD27, CD137 (4-1BB), and CD357 (glucocorticoid-induced tumor necrosis factor receptor family-related protein \[GITR\]) * Active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease \[e.g., ulcerative colitis, Crohn's disease\], celiac disease, or other serious chronic gastrointestinal conditions associated with diarrhea; autoimmune vasculitis; systemic lupus erythematosus; Wegener syndrome \[granulomatosis with polyangiitis\]; myasthenia gravis; Graves' disease; rheumatoid arthritis, hypophysitis, uveitis, etc.) within 3 years prior to the planned start of treatment; the following are exceptions to this criterion: * Vitiligo * Alopecia * Hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement * Psoriasis not requiring systemic treatment * Other conditions considered to be low risk of serious deterioration by the principal investigator (PI) * History of any one of the following cardiovascular conditions within the past 6 months: class III or IV heart failure as defined by the New York Heart Association (NYHA), cardiac angioplasty or stenting, myocardial infarction, or unstable angina; history of other clinically significant cardiac disease that, in the opinion of the PI or designee, is a contraindication to lymphodepleting chemotherapy, JCAR014 infusion, or durvalumab infusion is also excluded * History or presence of clinically relevant CNS pathology such as epilepsy, seizure, paresis, aphasia, stroke, severe brain injuries, dementia, Parkinson's disease, cerebellar disease, or psychosis; history of other organic brain syndrome that in the opinion of the PI or designee is a contraindication to lymphodepleting chemotherapy, JCAR014 infusion or durvalumab infusion * History of solid organ transplantation EXCLUSION CRITERIA FOR LYMPHODEPLETION CHEMOTHERAPY, JCAR014 AND DURVALUMAB: * For lymphodepletion chemotherapy, JCAR014 and durvalumab: Subjects with known active CNS involvement by malignancy; subjects with prior CNS disease that has been effectively treated will be eligible if treatment was completed at least 3 months prior to enrollment and there is no evidence of disease or stable abnormalities on repeat imaging * Uncontrolled infection * Receipt of live, attenuated vaccine within 28 days prior to the first dose of durvalumab (Note: enrolled patients should not receive live vaccine during the study and for 180 days after the last dose of durvalumab) * Planned use of corticosteroids (\> 10 mg/day prednisone or equivalent) or other systemic immunosuppression is not permitted within 72 hours prior to JCAR014 infusion; topical and/or inhaled steroids are permitted.