Type 2 diabetes mellitus is accompanied by cognitive impairment and is associated with an increased risk of dementia. Damage to brain structures such as white matter network disruption may underlie this cognitive disturbance. In the present study, 886 non-diabetic and 163 type 2 diabetic participants completed a battery of neuropsychological tests. Among them, 38 diabetic patients and 34 non-diabetic participants that matched the patients for age/sex/education received a magnetic resonance imaging-based diffusion tensor imaging. Then we calculated the topological properties of the white matter network using a graph theoretical method to investigate network efficiency differences between groups. We found that type 2 diabetic patients had inferior performances compared to the non-diabetic controls, in several cognitive domains involving executive function, spatial processing, memory, and attention. We also found that diabetic patients exhibited a disrupted topological organization of the white matter network (including the global network properties, i.e., network strength, global efficiency, local efficiency and shortest path length, and the nodal efficiency of the right rolandic operculum) in the brain. Moreover, those global network properties and the nodal efficiency of the right rolandic operculum both had positive correlations with executive function in the patient group. The results suggest that type 2 diabetes mellitus leads to an alteration in the topological organization of the cortical white matter network and this alteration may account for the observed cognitive decline.