Monoclonal Antibody Therapy in Treating Patients With Lymphoma or Colon Cancer That Has Not Responded to Vaccine Therapy

DISEASE CHARACTERISTICS: * Histologically confirmed prostate cancer (closed to accrual as of 3/9/2005) * Prior therapy on protocol NCI-00-C-0137 or NCI-00-C-0154 * Progressive disease (2 consecutively rising PSA levels, new bone scan lesion, or progression of soft tissue) * PSA at least 5 ng/mL * Progressive androgen-independent disease * Disease progression at least 4 weeks after flutamide withdrawal OR * Disease progression at least 6 weeks after bicalutamide or nilutamide withdrawal OR * Histologically confirmed follicular or mantle cell non-Hodgkin's lymphoma (mantle cell lymphoma closed to accrual as of 3/9/2005) * Prior therapy on protocol NCI-00-C-0133, NCI-01-C-0169, or NCI-00-C-0050 * Progressive disease after standard treatment * Relapsed disease OR * Histologically confirmed colon cancer (colon cancer closed to accrual as of 9/28/05) * Prior therapy on protocol NCI-99-C-0023 * Progressive disease OR * Histologically confirmed non-Hodgkin's lymphoma or Hodgkin's lymphoma * Progressive disease after standard treatment * No curative therapy exists * Prior allogeneic stem cell transplantation from a matched sibling or matched unrelated donor for an aggressive lymphoma allowed * Last infusion of allogeneic cells (either hematopoietic stem cells or donor lymphocytes) must have occurred \> 90 days prior to study enrollment * No other standard therapy available or refused such therapy * No symptomatic or rapidly progressive malignancy requiring therapy * No symptomatic CNS metastases PATIENT CHARACTERISTICS: Age * 18 and over Performance status * Karnofsky 80-100% Life expectancy * More than 2 months Hematopoietic * WBC at least 2,500/mm\^3 * Granulocyte count at least 1,500/mm\^3 * Platelet count at least 50,000/mm\^3 * Hemoglobin at least 10 g/dL * Hematocrit at least 30% Hepatic * Bilirubin no greater than 3.0 mg/dL (unless due to Gilbert's disease) * SGOT and SGPT no greater than 3 times upper limit of normal * Hepatitis B surface antigen negative * Hepatitis C antibody negative Renal * Creatinine no greater than 2.0 mg/dL Immunologic * HIV negative * Rheumatoid factor negative if history or evidence of arthritis * Anti-nuclear antibody (ANA) titer no greater than 1:80 if history or clinical signs or symptoms of connective tissue disease * No prior or active autoimmune disease (e.g., uveitis, rheumatoid arthritis, lupus erythematosus, autoimmune hemolytic anemia, ulcerative and hemorrhagic colitis, endocrine disorders \[e.g., thyroiditis, hyperthyroidism, hypothyroidism, autoimmune hypophysitis/hypopituitarism, or adrenal insufficiency\], sarcoid granuloma, myasthenia gravis, polymyositis,or Guillain-Barre syndrome) * No positive antibody titers to autoimmune diseases * Rheumatoid factor positive allowed unless ANA titer is greater than 1:80 and there is a history of or clinical signs or symptoms of connective tissue disease * No active infection Other * No other active malignancy within the past 5 years except adequately treated squamous cell or basal cell skin cancer, carcinoma in situ of the cervix, or superficial bladder cancer * Not pregnant or nursing * Negative pregnancy test * Fertile patients must use effective contraception during and for 4 months after study participation PRIOR CONCURRENT THERAPY: Biologic therapy * See Disease Characteristics * Recovered from prior vaccine therapy * No prior anti-cytotoxic T-lymphocyte-associated antigen-4 monoclonal antibody (MDX-CTLA4) for patients in part I of study * Patients in part II of study may have had up to 4 prior treatments with MDX-CTLA4 * No concurrent vaccine therapy * No concurrent infliximab Chemotherapy * At least 4 weeks since prior cytotoxic chemotherapy * No concurrent mercaptopurine, methotrexate, or cyclophosphamide Endocrine therapy * See Disease Characteristics * At least 4 weeks since prior steroids * No concurrent systemic, inhaled, or topical steroids Radiotherapy * At least 4 weeks since prior radiotherapy Surgery * At least 4 weeks since prior major surgery Other * Prior intervening therapy for prostate cancer, non-Hodgkin's lymphoma or colon cancer allowed * No other concurrent investigational therapy * No other concurrent immunosuppressants (e.g., cyclosporine or its analog)