Testing the Addition of Atezolizumab to Combination Chemotherapy or Atezolizumab Alone for Metastatic Colon or Rectal Cancer, the COMMIT Study

Inclusion Criteria: * The patient must have signed and dated an Institutional Review Board (IRB)-approved consent form that conforms to federal and institutional guidelines * Age \>= 18 years * Eastern Cooperative Oncology Group (ECOG) performance status of 0, 1 or 2 * Diagnosis of metastatic adenocarcinoma of colon or rectum without previous chemotherapy or any other systemic therapy for metastatic colorectal cancer except for one cycle of FOLFOX or capecitabine and oxaliplatin (CAPOX), either with or without bevacizumab prior to enrollment. Upon enrollment, the preceding single cycle of FOLFOX or FOLFOX + bevacizumab, if the patient received one, will not count towards patients' assessments per protocol. Cycle 1 day 1 (C1D1) of atezolizumab or C1D1 of mFOLFOX6/bevacizumab + atezolizumab will correspond to the first day the patient received therapy on trial * Tumor determined to be mismatch-repair deficient (dMMR) by Clinical Laboratory Improvement Act (CLIA)-certified immunohistochemical (IHC) assay with a panel of all four IHC markers, including MLH1, MSH2, PMS2, and MSH6; alternatively, MSI-H diagnosed by polymerase chain reaction (PCR)-based assessment of microsatellite alterations (either Bethesda markers or Pentaplex panel) or by next-generation sequencing (NGS) are eligible * Documentation by PET/CT scan, CT scan, or MRI that the patient has measurable metastatic disease per RECIST 1.1 * No immediate need for surgical intervention for the primary tumor or palliative diversion/bypass * Absolute neutrophil count (ANC) must be \>= 1500/mm\^3 (obtained within 28 days prior randomization) * Platelet count must be \>= 100,000/mm\^3 (obtained within 28 days prior randomization) * Hemoglobin must be \>= 8 g/dL (obtained within 28 days prior randomization) * Total bilirubin must be =\< 4 x ULN (upper limit of normal) (obtained within 28 days prior randomization); and * Aspartate aminotransferase (AST) and alanine aminotransferase (ALT) must be =\< 3 x ULN for the lab with the following exception: for patients with documented liver metastases, AST and ALT must be =\< 5 x ULN (obtained within 28 days prior randomization) * Calculated creatinine clearance \>= 30 mL/min (obtained within 28 days prior randomization) * A urine sample tested for proteinuria by either the dipstick method, urinalysis (UA), or a urine protein creatinine (UPC) ratio: * The dipstick method must indicate 0-1+ protein; if dipstick reading is \>= 2+, a 24-hour urine must be done and it must demonstrate \< 1.0 g of protein per 24 hours or a UPC ratio \< 1.0 * A urine protein creatinine (UPC) ratio must be \< 1.0; if the UPC ratio is \>= 1.0 a 24-hour urine must be done and it must demonstrate \< 1.0 g of protein per 24 hours * Urinalysis must indicate \< 30 mg/dl. If urinalysis \>= 30 mg/dl, a 24-hour urine must be done and it must demonstrate \< 1.0 g of protein per 24 hours or a UPC ratio \< 1.0 * International normalized ratio of prothrombin time (INR) and prothrombin time (PT) must be =\< 1.5 x ULN for the lab within 28 days before randomization; patients who are therapeutically treated with an agent such as warfarin may participate if they are on a stable dose and no underlying abnormality in coagulation parameters exists per medical history, regardless of PT/INR results * Pregnancy test done within 28 days prior randomization must be negative (for women of childbearing potential only); pregnancy testing should be performed according to institutional standards; administration of atezolizumab or mFOLFOX6/bevacizumab/atezolizumab may have an adverse effect on pregnancy and poses a risk to the human fetus, including embryo-lethality; should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately * Women of child-bearing potential and men must agree to use adequate contraception methods that result in a failure rate of \< 1% per year during the treatment period (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and for 5 months (150 days) after the last dose of atezolizumab, 6 months after the last dose of bevacizumab, and 6 months after the last dose of mFOLFOX6; a woman is considered to be of childbearing potential if she is not postmenopausal, has not reached a postmenopausal state (\>= 12 continuous months of amenorrhea with no identified cause other than menopause), and has not undergone surgical sterilization (removal of ovaries and/or uterus); examples of contraceptive methods with a failure rate of \< 1% per year include: bilateral tubal ligation; male partner sterilization; intrauterine devices; the reliability of sexual abstinence should be evaluated in relation to the duration of the clinical study and the preferred and usual lifestyle of the patient; periodic abstinence (e.g., calendar, ovulation, symptothermal, or postovulation methods) and withdrawal are not acceptable methods of contraception; men must refrain from donating sperm during this same period Exclusion Criteria: * Known hypersensitivity to Chinese hamster ovary cell products or other recombinant human antibodies, fluoropyrimidines, folic acid derivatives or oxaliplatin * Uncontrolled high blood pressure defined as systolic blood pressure (BP) \> 150 mmHg or diastolic BP \> 100 mmHg with or without anti-hypertensive medication; patients with initial BP elevations are eligible if initiation or adjustment of BP medication lowers pressure to meet entry criteria * Documented New York Heart Association (NYHA) class III or IV congestive heart failure * Serious or non-healing wound, skin ulcer, or bone fracture * History of inherited bleeding diathesis, gastrointestinal (GI) perforation, significant vascular disease (e.g., aortic aneurysm requiring surgical repair or recent arterial thrombosis or symptomatic peripheral ischemia, transient ischemic attack \[TIA\], cerebrovascular accident \[CVA\] or arterial thrombotic event), abdominal fistula, intra-abdominal abscess, or active GI bleeding (with cause not addressed) within 6 months prior to randomization, or other medical condition in the opinion of the treating oncologist that makes the risk of cardiovascular or bleeding complications with bevacizumab use unacceptably high * Other malignancies are excluded unless the patient has completed therapy for the malignancy \>= 12 months prior to randomization and is considered disease-free; patients with the following cancers are eligible if diagnosed and treated within the past 12 months: in situ carcinomas or basal cell and squamous cell carcinoma of the skin * Known DPD (dihydro pyrimidine dehydrogenase) deficiency * Symptomatic peripheral sensory neuropathy \>= grade 2 (Common Terminology Criteria for Adverse Events \[CTCAE\] version \[v\] 5.0) * Prior treatment with oxaliplatin chemotherapy within 6 months prior to randomization * History of grade 2 hemoptysis (defined as 2.5 mL of bright red blood per episode) within 1 month prior to screening * Prior treatment with anti-PD-1, or anti-PD-L1 therapeutic antibody or pathway-targeting agents; patients who have received prior treatment with anti-CTLA-4 may be enrolled provided the following requirements are met: * Minimum of 12 weeks from the first dose of anti-CTLA-4 and \> 6 weeks from the last dose to randomization * No history of severe immune-related adverse effects (CTCAE grade 3 and 4) from anti-CTLA-4 * Treatment with systemic immunosuppressive medications (including, but not limited to, prednisone, cyclophosphamide, azathioprine, methotrexate, thalidomide, and anti-tumor necrosis factor \[anti-TNF\] agents) within 14 days prior to randomization; however, * Patients who have received acute, low dose, systemic immunosuppressant medications (e.g., a one-time dose of dexamethasone for nausea; or chronic daily treatment with corticosteroids with a dose of =\< 10 mg/day methylprednisolone equivalent) may be enrolled * The use of inhaled corticosteroids and mineralocorticoids (e.g., fludrocortisone) for patients with orthostatic hypotension or adrenocortical insufficiency is allowed * Known clinically significant liver disease, including active viral, alcoholic, or other hepatitis; cirrhosis; fatty liver; and inherited liver disease; however, * Patients with past or resolved hepatitis B infection (defined as having a negative hepatitis B surface antigen \[HbsAg\] test and a positive anti-HBc \[antibody to hepatitis B core antigen\] antibody test) are eligible if polymerase chain reaction (PCR) for hepatits B virus (HBV) ribonucleic acid (RNA) is negative per local guidelines * Patients positive for hepatitis C virus (HCV) antibody are eligible only if polymerase chain reaction (PCR) is negative for HCV RNA per local guidelines * History or risk of autoimmune disease, including, but not limited to, systemic lupus erythematosus, rheumatoid arthritis, inflammatory bowel disease, vascular thrombosis associated with antiphospholipid syndrome, Wegener's granulomatosis, Sjogren's syndrome, Bell's palsy, Guillain-Barre syndrome, multiple sclerosis, autoimmune thyroid disease, vasculitis, or glomerulonephritis; however, * Patients with a history of autoimmune hypothyroidism on a stable dose of thyroid replacement hormone may be eligible * Patients with controlled type 1 diabetes mellitus on a stable insulin regimen may be eligible * Patients with eczema, psoriasis, lichen simplex chronicus or vitiligo with dermatologic manifestations only (e.g., patients with psoriatic arthritis would be excluded) are permitted provided that they meet the following conditions: * Patients with psoriasis must have a baseline ophthalmologic exam to rule out ocular manifestations * Rash must cover less than 10% of body surface area (BSA) * Disease is well controlled at baseline and only requiring low potency topical steroids (e.g., hydrocortisone 2.5%, hydrocortisone butyrate 0.1%, fluocinolone 0.01%, desonide 0.05%, alclometasone dipropionate 0.05%) * No acute exacerbations of underlying condition within the last 12 months (not requiring psoralen plus ultraviolet A radiation \[PUVA\], methotrexate, retinoids, biologic agents, oral calcineurin inhibitors; high potency or oral steroids) * History of idiopathic pulmonary fibrosis, organizing pneumonia (i.e., bronchiolitis obliterans, cryptogenic organizing pneumonia, etc.), or active or recently active (within 90 days of randomization) pneumonitis (including drug induced) that required systemic immunosuppressive therapy (i.e. corticosteroids, etc.). History of radiation pneumonitis in the radiation field (fibrosis) is permitted * History of severe allergic, anaphylactic, or other hypersensitivity reactions to chimeric or humanized antibodies or fusion proteins * Patients with known active tuberculosis (TB) are excluded * Severe infections within 28 days prior to randomization, including but not limited to, hospitalization for complications of infection, bacteremia, or severe pneumonia * Signs or symptoms of infection within 14 days prior to randomization * Received oral or intravenous (IV) antibiotics within 14 days prior to randomization; patients receiving prophylactic antibiotics (e.g., for prevention of a urinary tract infection or chronic obstructive pulmonary disease) are eligible * Major surgical procedure, open biopsy, or significant traumatic injury within 28 days prior to randomization or anticipation of need for a major surgical procedure during the course of the study * The administration of a live, attenuated vaccine within 28 days prior to randomization * Pregnant women are excluded from this study because atezolizumab is an agent with the potential for teratogenic or abortifacient effects; because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with atezolizumab, breastfeeding should be discontinued if the mother is treated with atezolizumab; these potential risks may also apply to other agents used in this study; (Note: pregnancy testing should be performed within 28 days prior to randomization according to institutional standards for women of childbearing potential) * Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial * Patients with prior allogeneic bone marrow transplantation or prior solid organ transplantation