GDC-0449 and Erlotinib Hydrochloride With or Without Gemcitabine Hydrochloride in Treating Patients With Metastatic Pancreatic Cancer or Solid Tumors That Cannot Be Removed by Surgery

Inclusion Criteria: * Histologic proof of a solid tumor that is now unresectable, not amenable to any other standard therapies, or patient refuses standard therapy * Metastatic adenocarcinoma of the pancreas amenable to biopsies (cohort II MTD only) * Absolute neutrophil count (ANC) \>= 1,500/μL * Platelets \>= 100,000/μL * Total bilirubin =\< upper limit of normal (ULN) * Aspartate aminotransferase (AST) =\< 3 times upper limit of normal (ULN) * Creatinine =\< 1.5 times ULN * Hemoglobin \>= 9.0 g/dL * International Normalized Ratio (INR) within normal limits (for patients treated at the MTD) * Ability to provide informed consent * Willingness to return to Mayo Clinic for follow up * Life expectancy \>= 12 weeks * Willingness to provide the biologic specimens as required by the protocol * Negative serum pregnancy test done =\< 7 days prior to registration * Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0-2 * Able to swallow or have medication administered through a G-tube and absorb the medication * Participant agrees to use an acceptable form of contraception; acceptable forms of contraception: * Latex condom (always used with spermicide) * Diaphragm (always used with spermicide) * Cervical cap (always used with spermicide) Acceptable forms of secondary contraceptions, when used along with a barrier method: * Hormonal contraception methods, including pills, patches, rings, or injections except progestin-only containing pills (i.e. "Mini-pill") * Tubal ligation * Partner's vasectomy * Intrauterine device (IUD) (non-progesterone T) * Vaginal sponge (containing spermicide) * 100% commitment to abstinence Unacceptable forms of contraception for women of childbearing potential: * Oral contraception containing progestins only * IUD progesterone T * Female condom * Natural family planning (rhythm method) or breastfeeding * Fertility awareness * Withdrawal * Cervical shield * Willing to abstain from smoking * Willing to complete a daily pill diary Exclusion Criteria: * Known standard therapy for the patient's disease that is potentially curative or definitely capable of extending life expectancy * Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements * Any of the following prior therapies: * Chemotherapy =\< 4 weeks prior to registration * Mitomycin C/nitrosoureas =\< 6 weeks prior to registration * Immunotherapy =\< 4 weeks prior to registration * Biologic therapy =\< 4 weeks prior to registration * Radiation therapy =\< 4 weeks prior to registration * Radiation to \> 25% of bone marrow * Failure to fully recover from acute, reversible effects of prior therapy regardless of interval since last treatment * New York Heart Association classification III or IV * Seizure disorder * Central nervous system (CNS) metastases if not stable for at least 2-3 months based on imaging, clinical assessment, and use of steroids, or seizure disorder * Pregnant women * Nursing women * Men or women of childbearing potential who are unwilling to employ adequate contraception until 12 months after last study drug dose * Other concurrent chemotherapy, immunotherapy, radiotherapy, or any ancillary therapy considered investigational (utilized for a non-Food and Drug Administration \[FDA\]-approved indication and in the context of a research investigation) * Current therapy with a CYP3A4 inhibitor or inducer * Immunocompromised patients (other than that related to the use of corticosteroids) including patients receiving highly active antiretroviral therapy (HAART) treatment * Receiving any other investigational agent which would be considered as a treatment for the primary neoplasm * Active other malignancy, excepting non-melanotic skin cancer or carcinoma-in-situ of the cervix; if there is a history of prior malignancy, they must not be receiving other specific treatment (other than hormonal therapy) for their cancer * History of myocardial infarction =\< 6 months, or congestive heart failure requiring use of ongoing maintenance therapy for life-threatening ventricular arrhythmias * Abnormalities of the cornea based on history (e.g., dry eye syndrome, Sjogren's syndrome), congenital abnormality (e.g., Fuch's dystrophy), abnormal slit-lamp examination using a vital dye (e.g., fluorescein, Bengal Rose), and/or an abnormal corneal sensitivity test (Schirmer test or similar tear production test) * More than 2 prior chemotherapy regimens for the current metastatic malignancy; full dose chemotherapy used in conjunction with concurrent radiation therapy will be included as prior therapy; Note: prior hormonal therapy (e.g. leuprolide, aromatase inhibitors, tamoxifen) will be allowed and not included as a prior chemotherapy * Previous therapy with a hedgehog inhibitor