Subjective cognitive decline (SCD) is characterized by self-reported cognitive deficits without measurable cognitive impairment. It has been suggested that individuals with SCD exhibited brain structural alterations in widespread cortical thinning or gray matter loss in the medial temporal and frontotemporal regions. Apolipoprotein E (APOE) ε4 allele is thought to be a genetic marker associated with risk of SCD. Neuropsychiatric symptoms may provide insight in detecting higher-risk elders for early Alzheimer's disease as well. Therefore, we aim to explore the characteristics of brain morphology in SCD and to determine whether it is influenced by APOE ε4 as well as neuropsychiatric symptoms in SCD. A total of 138 cognitively normal older individuals from the SILCODE cohort underwent a clinical interview, neuropsychological assessments, a blood test, and MRI. A two-sample t-test was used to examine the cortex volume and bilateral cortical surface area alterations between SCD (n = 65) and controls (n = 73). A general linear model analysis was used to test for both main and interaction effects of clinical phenotype (SCD vs. controls) and APOE on global and regional cortex volume and bilateral cortical surface area and thickness. A multiple linear regression analysis was conducted to determine the effects of the APOE genotype on the relationships between morphometric features and neuropsychiatric symptoms in SCD. Compared with controls, individuals with SCD showed decreased total cortical volumes and cortical surface area. SCD APOE ε4 carriers showed additive reduction in the right cortical surface area. The evaluation scores of anxiety symptoms were negatively associated with the right cortical surface area in SCD APOE 4 non-carriers. Individuals with SCD had an altered cortical surface area, and APOE genotype and anxiety symptoms are modified factors on the cortical surface area decrease in SCD. ClinicalTrials.gov (Identifier: NCT03370744 ). Registered 15 March 2017.