Colorectal cancer starts in the innermost layer (the mucosa) and can grow outward through some or all of the other layers (see picture below). 
Colorectal cancer starts in the inner layer called the mucosa, and can grow outward through some or all of the other layers of the colon (see picture below). 