Background and purpose: SORL1 rs1699102 is associated with the risk of late-onset Alzheimer's disease. However, the effects of this single nucleotide polymorphism on cognition and brain structure during normal aging are unclear. This study aimed to examine the effects of the rs1699102 polymorphism on age-related cognitive decline and cortical gray matter reduction in the Chinese Han population. Methods: A total of 780 non-demented adults completed a battery of neuropsychological tests. High-resolution T1-weighted structural magnetic resonance imaging data from 89 of these subjects were also collected using a Siemens Trio 3.0 Tesla scanner. Results: The T allele carriers displayed an accelerated age-related change in episodic memory and processing speed tests relative to the CC genotype. A similar pattern was observed in the age-related gray matter volume (GMV) reduction of the right middle temporal pole. The GMV in this region was significantly positively correlated with the episodic memory scores. Conclusions: The SORL1 gene rs1699102 polymorphism has been found to be associated with age-related cognitive decline and GMV reduction of the right middle temporal pole in older adults. These findings elucidate how the SORL1 variants shape the neural system to modulate age-related cognitive decline and support the hypothesis that SORL1 may represent a candidate gene for late-onset Alzheimer's disease.