We explored changes in clinical features and neuropathological mechanisms underlying olfactory dysfunction (OD) in 60 patients with Alzheimer's disease (AD). Olfactory function was evaluated using the Sniffin' Sticks test and a threshold discrimination identification (TDI) score. Based on the TDI score, we divided patients according to the presence or absence of OD (AD-OD and AD-NOD, respectively). Cognitive and neuropsychiatric symptoms were evaluated by a series of rating scales. The volumes and cortical thickness of the thalamus, hippocampus, and amygdala were measured using structural magnetic resonance imaging. Neuropathological protein levels in cerebrospinal fluid were measured. The frequency of OD was 50%. TDI scores were lower in the AD-OD group than in the AD-NOD group (p < 0.001). Compared with the AD-NOD group, the AD-OD group showed greater cognitive function impairments (p < 0.001), and daily living activities were more severely compromised (p = 0.019). The AD-OD group had lower hippocampal and amygdala volumes (p = 0.025, p = 0.030, respectively) and a more pronounced reduction in cortical thickness (p = 0.010). The total tau level was lower in the AD-OD group than the AD-NOD group (p = 0.040). Lower Mini-Mental State Examination scores and thinner AD-signature cortices were associated with lower TDI scores (OR = 0.826, p < 0.001; OR = 1.433, p = 0.008). Overall, in AD patients, the impairments in olfactory discrimination and identification seem to be more correlated with cognitive levels. OD in AD may be an indicator of pathological cognitive decline and structural changes.