Nivolumab and Temozolomide in Treating Patients With Recurrent or Refractory Small-Cell Lung Cancer or Advanced Neuroendocrine Cancer

Inclusion Criteria: * Be willing and able to provide written informed consent/assent for the trial * For Cohort 1: Have histologically or cytologically-documented diagnosis of advanced (metastatic and/or unresectable) small cell lung cancer and have progressed or recurred after platinum-based chemotherapy with immunotherapy. Eligible patients will be defined as follows: * Sensitive disease: Patients who had one previous line of chemotherapy and relapsed after \> 90 days of completion of treatment * Refractory disease: Patients with no response to first-line chemotherapy or progression \< 90 days after completing treatment. * For cohort 1: maximum of 2 prior lines of therapy is allowed (ie second or third line treatment) * For Cohort 2: Have histologically or cytologically-documented diagnosis of advanced (metastatic and/or unresectable) neuroendocrine tumor/carcinoma of any grade (World Health Organization \[WHO\] Grade 1-3) of any origin, in any line of therapy (ie both treatment na?ve and pre-treated patients allowed), and have clinical or biochemical or radiographic progression in the 12 months prior to study registration. Concomitant use of a somatostatin analogue is allowed, as long as patients have been on a stable dose for at least 2 months * Have measurable disease based on Response Evaluation Criteria in Solid Tumors (RECIST) 1.1. * For Cohort 1: Availability of a recent formalin-fixed, paraffin-embedded (FFPE) tumor tissue block. A recently obtained archival FFPE tumor tissue block from a primary or metastatic tumor resection or biopsy can be provided if it was obtained within 1 year of trial screening. Patients with tumor specimens older than 1 year may still be eligible if deemed so by study sponsor. For Cohort 2: archival tissue as above is preferred, but not required for trial entry. Verification of tumor burden in the biopsy is encouraged. For optimal biomarker results, tumor content should be \> 30% of total tissue area * Be willing to provide peripheral blood samples at screening and day 1 of cycles 1, 2 and 3 as well as at progressive disease for correlative studies * Have a performance status of 0 or 1 on the Eastern Cooperative Oncology Group (ECOG) performance scale * Life expectancy greater than 3 months * Ability to swallow and retain oral medication * Absolute neutrophil count (ANC) \>= 1,500 /mcL (performed within 28 days of treatment initiation) * Platelets \>= 100,000 / mcL (performed within 28 days of treatment initiation) * Serum creatinine =\< 2.0 x upper limit of normal (ULN) OR measured or calculated creatinine clearance (CrCl) (glomerular filtration rate \[GFR\] can also be used in place of creatinine or CrCl) \>= 50 mL/min as estimated by Cockcroft and Gault formula for subject with creatinine levels \> 2 X institutional ULN (performed within 28 days of treatment initiation) * Creatinine clearance should be calculated per institutional standard * Serum total bilirubin =\< 1.5 x ULN OR direct bilirubin =\< ULN for subjects with total bilirubin levels \> 1.5 ULN (performed within 28 days of treatment initiation) * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\]) and Alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x ULN OR =\< 5 x ULN for subjects with liver metastases (performed within 28 days of treatment initiation) * Both values must be in the specified range * Albumin \>= 2.5 mg/dL (performed within 28 days of treatment initiation) * International normalized ratio (INR) or prothrombin time (PT) =\< 1.5 x ULN unless subject is receiving anticoagulant therapy as long as PT or partial thromboplastin time (PTT) is within therapeutic range of intended use of anticoagulants (performed within 28 days of treatment initiation) * Activated partial thromboplastin time (aPTT) =\< 1.5 x ULN unless subject is receiving anticoagulant therapy as long as PT or PTT is within therapeutic range of intended use of anticoagulants (performed within 28 days of treatment initiation) * Female subject of childbearing potential should have a negative urine or serum pregnancy within 72 hours prior to receiving the first dose of study medication. If the urine test is positive or cannot be confirmed as negative, a serum pregnancy test will be required * Female subjects of childbearing potential should be willing to use 2 methods of birth control or be surgically sterile, or abstain from heterosexual activity and not donate eggs for the course of the study through 120 days after the last dose of study medication. Subjects should agree to ongoing pregnancy testing during the course of the study and after the end of study therapy. Female subjects of childbearing potential are those who have not been surgically sterilized or have not been free from menses for \> 1 year Exclusion Criteria: * Is currently participating and receiving study therapy or has participated in a study of an investigational agent and received study therapy or used an investigational device within 4 weeks of the first dose of treatment * Has received prior temozolomide therapy * Prior immunotherapy with checkpoint inhibitors (including antibodies to PD-1, PD-L1, is allowed only in Cohort 1 and must have been given in combination with chemotherapy as part of first line treatment. Prior CTLA-4 therapy is excluded in both cohorts. * Has a diagnosis of immunodeficiency or is receiving systemic steroid therapy or any other form of immunosuppressive therapy within 7 days prior to the first dose of trial treatment * Has a known history of active TB (Bacillus Tuberculosis) * Hypersensitivity to nivolumab or temozolomide or any of their excipients * Has had prior chemotherapy, targeted small molecule therapy, or radiation therapy within 2 weeks prior to study day 1 or who has not recovered (i.e., =\< grade 1 or at baseline) from adverse events due to a previously administered agent * Note: Subjects with =\< grade 2 neuropathy due to chemotherapy are an exception to this criterion and may qualify for the study * Note: If subject received major surgery, they must have recovered adequately from the toxicity and/or complications from the intervention prior to starting therapy * Has a known additional malignancy that is progressing or requires active treatment * Has symptomatic central nervous system (CNS) metastases and/or carcinomatous meningitis. Subjects with asymptomatic lesions will be eligible if considered appropriate by the treating physician. Subjects with previously treated brain metastases may participate provided they are stable (without evidence of progression by imaging for at least two weeks prior to the first dose of trial treatment and any neurologic symptoms have returned to baseline), have no evidence of new or enlarging brain metastases, and are not requiring steroids (or are on a stable or decreasing dose of steroids equivalent to 10 mg prednisone or less) for at least 7 days prior to trial treatment. This exception does not include carcinomatous meningitis which is excluded regardless of clinical stability * Has active autoimmune disease, including myasthenic syndrome, which has required systemic treatment in the past 2 years (i.e. with use of disease modifying agents, corticosteroids or immunosuppressive drugs). Replacement therapy (eg., thyroxine, insulin, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment * Has an active infection requiring systemic therapy * Has a history or current evidence of any condition, therapy, or laboratory abnormality that might confound the results of the trial, interfere with the subject?s participation for the full duration of the trial, or is not in the best interest of the subject to participate, in the opinion of the treating investigator * Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial * Is pregnant or breastfeeding, or expecting to conceive within the projected duration of the trial, starting with the pre-screening or screening visit through 5 months after the last dose of trial treatment * Has a known history of human immunodeficiency virus (HIV) (HIV 1/2 antibodies) * Has known active hepatitis B (e.g., hepatitis B surface antigen \[HBsAg\] reactive) or hepatitis C (e.g., hepatitis C virus \[HCV\] ribonucleic acid \[RNA\] \[qualitative\] is detected) * Any condition which in the Investigator?s opinion deems the participant an unsuitable candidate to receive study drug