The tumor environment is of vital importance for the incidence and development of colorectal cancer. Increasing evidence in recent years has elaborated the vital role of the tumor environment in cancer subtype classification and patient prognosis, but a comprehensive understanding of the colorectal tumor environment that is purely dependent on the stromal compartment is lacking. To decipher the tumor environment in colorectal cancer and explore the role of its immune context in cancer classification, we performed a gene expression microarray on the stromal compartment of colorectal cancer and adjacent normal tissues. Through the integrated analysis of our data with public gene expression microarray data of stromal and epithelial colorectal cancer tissues processed through laser capture microdissection, we identified four highly connected gene modules representing the biological features of four tissue compartments by applying a weighted gene coexpression network analysis algorithm and classified colorectal cancers into three immune subtypes by adopting a nearest template prediction algorithm. A systematic analysis of the four identified modules mainly reflected the close interplay between the biological changes of intrinsic and extrinsic characteristics at the initiation of colorectal cancer. Colorectal cancers were stratified into three immune subtypes based on gene templates identified from representative gene modules of the stromal compartment: active immune, active stroma, and mixed type. These immune subtypes differed by the immune cell infiltration pattern, expression of immune checkpoint inhibitors, mutation landscape, extent of mutation burden, extent of copy number burden, prognosis and chemotherapeutic sensitivity. Further analysis indicated that activation of the NF-kB signaling pathway was the major mechanism causing the no immune infiltration milieu in the active stroma subtype and that inhibitors of the NF-kB signaling pathway could be candidate drugs for treating patients with an active stroma. Overall, these results suggest that characterizing colorectal cancer by the tumor environment is of vital importance in predicting patients' clinical outcomes and helping guide precision and personalized treatment. Copyright © 2020 Shen, Li, Li, Zhang, Feng and Cheng.