The topological architecture of the cerebral anatomical network reflects the structural organization of the human brain. Recently, topological measures based on graph theory have provided new approaches for quantifying large-scale anatomical networks. However, few studies have investigated the hemispheric asymmetries of the human brain from the perspective of the network model, and little is known about the asymmetries of the connection patterns of brain regions, which may reflect the functional integration and interaction between different regions. Here, we utilized diffusion tensor imaging to construct binary anatomical networks for 72 right-handed healthy adult subjects. We established the existence of structural connections between any pair of the 90 cortical and subcortical regions using deterministic tractography. To investigate the hemispheric asymmetries of the brain, statistical analyses were performed to reveal the brain regions with significant differences between bilateral topological properties, such as degree of connectivity, characteristic path length, and betweenness centrality. Furthermore, local structural connections were also investigated to examine the local asymmetries of some specific white matter tracts. From the perspective of both the global and local connection patterns, we identified the brain regions with hemispheric asymmetries. Combined with the previous studies, we suggested that the topological asymmetries in the anatomical network may reflect the functional lateralization of the human brain.