Ipilimumab and Gemcitabine Hydrochloride in Treating Patients With Stage III-IV or Recurrent Pancreatic Cancer That Cannot Be Removed by Surgery

Inclusion Criteria: * Willing and able to give written informed consent * Histologic or cytologic diagnosis of pancreas adenocarcinoma advanced or recurrent (stage III or IV) that is unresectable; histologic or cytologic pathology from any prior surgery is sufficient for diagnosis * Must have measurable disease by modified WHO criteria * White blood cells (WBC) \>= 2000/uL * Absolute neutrophil count (ANC) \>= 1500/uL * Platelets \>= 100 x 10\^3/uL * Hemoglobin \>= 9 g/dL (\>= 80 g/L; may be transfused) * Creatinine =\< 2.0 x upper limit of normal (ULN) * Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 2.5 x ULN * Bilirubin =\< 2.0 x ULN, (except patients with Gilbert's Syndrome, who must have a total bilirubin less than 3.0 mg/dL) * No active or chronic infection with human immunodeficiency virus (HIV), hepatitis B, or hepatitis C * Performance status: Eastern Cooperative Oncology Group (ECOG) 0-1 * Prior systemic therapy for advanced pancreas cancer with gemcitabine is prohibited; prior gemcitabine with radiotherapy for localized pancreas cancer is allowed provided disease is present outside of the radiated field; prior gemcitabine as adjuvant therapy to surgical resection is allowed provided 3 months or greater has elapsed between the last dose of gemcitabine and the detection of recurrent disease * Women of childbearing potential (WOCBP) must be using an adequate method of contraception to avoid pregnancy throughout the study and for up to 26 weeks after the last dose of investigational product, in such a manner that the risk of pregnancy is minimized; WOCBP include any female who has experienced menarche and who has not undergone successful surgical sterilization (hysterectomy, bilateral tubal ligation, or bilateral oophorectomy) or is not post-menopausal; post-menopause is defined as: amenorrhea \>= 12 consecutive months without another cause, or for women with irregular menstrual periods and taking hormone replacement therapy (HRT), a documented serum follicle stimulating hormone (FSH) level \>= 35 mIU/mL; women who are using oral contraceptives, other hormonal contraceptives (vaginal products, skin patches, or implanted or injectable products), or mechanical products such as an intrauterine device or barrier methods (diaphragm, condoms, spermicides) to prevent pregnancy, or are practicing abstinence or where their partner is sterile (eg, vasectomy) should be considered to be of childbearing potential * WOCBP must have a negative serum or urine pregnancy test (minimum sensitivity 25 IU/L or equivalent units of human chorionic gonadotrophin \[HCG\]) within 72 hours before the start of ipilimumab * Men of fathering potential must be using an adequate method of contraception to avoid conception throughout the study (and for up to 26 weeks after the last dose of investigational product) in such a manner that the risk of pregnancy is minimized * Patients on stable anticoagulation are eligible for enrollment; for patients on warfarin, prothrombin time (PT)/international normalized ratio (INR) should be monitored every 2 weeks during induction therapy, monthly thereafter, or more frequent as clinically indicated Exclusion Criteria: * Any other malignancy from which the patient has been disease-free for less than 5 years, with the exception of adequately treated and cured basal or squamous cell skin cancer, superficial bladder cancer or carcinoma in situ of the cervix * Autoimmune disease: Patients with a history of inflammatory bowel disease, including ulcerative colitis and Crohn's Disease, are excluded from this study, as are patients with a history of symptomatic disease (eg, rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[eg, Wegener's Granulomatosis\]); motor neuropathy considered of autoimmune origin (e.g. Guillain-Barre Syndrome and Myasthenia Gravis) * Any underlying medical or psychiatric condition, which in the opinion of the investigator will make the administration of ipilimumab hazardous or obscure the interpretation of adverse events (AEs), such as a condition associated with frequent diarrhea * Any non-oncology vaccine therapy used for prevention of infectious diseases (for up to 1 month before or after any dose of ipilimumab) * A history of prior treatment with ipilimumab or prior tumor necrosis factor receptor superfamily, member 9 (CD137) agonist or cytotoxic T-lymphocyte-associated protein 4 (CTLA4) inhibitor or agonist * Concomitant therapy with any of the following: interleukin (IL)2, interferon, or other non-study immunotherapy regimens; immunosuppressive agents; other investigation therapies; or chronic use of systemic corticosteroids * WOCBP who are unwilling or unable to use an acceptable method of contraception to avoid pregnancy for their entire study period and for at least 8 weeks after cessation of study drug, or have a positive pregnancy test at baseline, or are pregnant or breastfeeding * Prisoners or subjects who are compulsorily detained (involuntarily incarcerated) for treatment of either a psychiatric or physical (eg, infectious) illness * Patients with symptoms of partial or complete bowel obstruction and recent (within 6 month) history of fistula, intra-abdominal abscess or bowel perforation * Patients with a history or evidence of central nervous system (CNS) disease, including brain tumor, seizures not controlled with standard medical therapy or any brain metastases * Patients currently receiving radiation therapy or those having received radiation within 4 weeks of study entry * Patients with any known active infection or known history of tuberculosis