Background Neuroendocrine carcinomas of the gastrointestinal tract (GI-NECs) remain a disease of grim prognosis with limited therapeutic options. Their molecular characteristics are still undefined. This study aimed to explore the underlying genetic basis and heterogeneity of GI-NECs. Methods Comprehensive genomic analysis using whole-exome sequencing was performed on 143 formalin-fixed, paraffin-embedded samples of surgically resected GI-NEC with a thorough histological evaluation. Mutational signatures, somatic mutations, and copy number aberrations were analyzed and compared across anatomic locations and histological subtypes. Survival analysis was conducted to identify the independent factors. Results In total, 143 GI-NECs were examined: the stomach, 87 cases (60.8%); the esophagus, 29 cases (20.3%); the colorectum, 20 cases (14.0%); and the small intestine, 7 cases (4.9%). Eighty-three (58.0%) and 60 (42.0%) cases were subclassified into small cell and large cell subtypes, respectively. GI-NECs showed distinct genetic alterations from their lung counterparts and non-neuroendocrine carcinomas in the same locations. Obvious heterogeneity of mutational signatures, somatic mutations, and copy number variations was revealed across anatomic locations rather than histological subtypes. Except for tumor protein p53 (TP53) and retinoblastoma 1 (RB1), the most frequently mutated genes in the stomach, esophagus, colorectum, and small intestine were low-density lipoprotein receptor-related protein 1B (LRP1B), notch receptor 1 (NOTCH1), adenomatosis polyposis coli (APC), catenin beta 1 (CTNNB1), respectively. Mutations in the WNT-beta-catenin, NOTCH and erythroblastic leukemia viral oncogene B (ERBB) pathways were prevalently identified in gastric, esophageal, and colorectal NECs, respectively. Importantly, 104 (72.7%) GI-NECs harbored putative clinically relevant alterations, and non-gastric location and RB1 bi-allelic inactivation with copy number alterations were identified as two independent poor prognostic factors. Furthermore, we found that tumor cells in GI-NECs first gain clonal mutations in TP53, RB1, NOTCH1 and APC, followed by subsequent whole-genome doubling (WGD) and post-WGD clonal mutations in LRP1B, CUB and Sushi multiple domains 3 (CSMD3), FAT tumor suppressor homolog 4 (FAT4) and erb-b2 receptor tyrosine kinase 4 (ERBB4), and finally develop subclonal mutations. Conclusions GI-NECs harbor distinct genomic landscapes and demonstrate significant genetic heterogeneity across different anatomic locations. Moreover, potentially actionable alterations and prognostic factors were revealed for GI-NECs.