A high-fat, high-energy (HFE) diet may be deleterious to the cardiovascular system and mental health. We previously reported that serum cholesterol levels and escape latency were significantly increased in mice by feeding them an HFE diet from gestation onward. In this study, we examined whether an HFE diet supplemented with phytosterols fed to pregnant C57BL/6j dams and their offspring would protect the HFE-diet-induced compromise of the offspring's learning capability. We measured serum cholesterol levels, brain N-methyl-D-aspartate receptor (NMDAR1) mRNA and protein expression and liver sterol 27-hydroxylase (Cyp27a1) mRNA expression, as well as a Morris water maze performance. The results showed that, compared to mice consuming the HFE diet alone, those also consuming phytosterols (the HFE + PS diet) significantly decreased mean serum low-density lipoprotein cholesterol levels and altered brain NMDAR1 mRNA and protein expression and liver Cyp27a1 mRNA expression. The Morris water maze experiments indicated that dietary phytosterol supplementation slightly decreased the escape latency (p = 0.07). Collectively, these observations suggest that consumption of phytosterols from early in life may help alleviate the detrimental effects of HFE diets in mice.