Introduction Vascular calcification (VC) is an independent risk factor for cardiovascular mortality in end-stage renal disease (ESRD) patients. The pathogenesis of VC is complicated and unclear. Uremic toxins produced by gut microbiota can promote VC. This study aims to identify the differences in gut microbiota between the different VC groups and the main bacteria associated with VC in hemodialysis (HD) patients in an attempt to open up new preventive and therapeutic approaches and define the probable mechanism for VC in HD patients in the future. Methods A total of 73 maintenance HD patients were enrolled in this cross-sectional study. According to the abdominal aortic calcification (AAC) scores, the participants were divided into the high AAC score group and the low AAC score group. High-throughput sequencing of the gut microbiota was performed and the results were evaluated by alpha diversity, beta diversity, species correlation, and model predictive analyses. Results The prevalence of VC was 54.79% (40/73) in the study. The majority of phyla in the two groups were the same, including Firmicutes, Actinobacteriota, Proteobacteria, and Bacteroidota. The microbial diversity in the high AAC score group had a decreasing trend (p = 0.050), and the species abundance was significantly lower (p = 0.044) than that in the low AAC score group. The HD patients with high AAC scores showed an increased abundance of Proteobacteria and decreased abundances of Bacteroidota and Synergistota at the phylum level; increased abundances of Escherichia-Shigella, Ruminococcus_gnavus_group, and Lactobacillus; and decreased abundances of Ruminococcus and Lachnospiraceae_NK4A136_group at the genus level (p<0.05). Escherichia-Shigella and Ruminococcus_gnavus_group were positively correlated with VC, and Ruminococcus, Adlercreutzia, Alistipes, and norank_f__Ruminococcaceae were negatively correlated with VC. Escherichia-Shigella had the greatest influence on VC in HD patients, followed by Ruminococcus and Butyricimonas. Conclusions Our results provide clinical evidence that there was a difference in gut microbiota between the different VC groups in HD patients. Escherichia-Shigella, a lipopolysaccharide (LPS)-producing bacterium, was positively correlated with VC and had the greatest influence on VC. Ruminococcus, a short-chain fatty acid (SCFA)-producing bacterium, was negatively correlated with VC and had the second strongest influence on VC in HD patients. The underlying mechanism is worth studying. These findings hint at a new therapeutic target.