Background and objective: Brain region volume from Structural Magnetic Resonance Imaging (sMRI) can directly reflect abnormal states in brain aging. While promising for clinical brain health assessment, existing volume-based brain age prediction methods fail to explore both linear and nonlinear relationships, resulting in weak representation and suboptimal estimates. Methods: This paper proposes a brain age prediction method, RFBLSO, based on Random Forest (RF), Broad Learning System (BLS), and Leave-One-Out Cross Validation (LOO). Firstly, RF is used to eliminate redundant brain regions with low correlation to the target value. The objective function is constructed by integrating feature nodes, enhancement nodes, and optimal regularization parameters. Subsequently, the pseudo-inverse method is employed to solve for the output coefficients, which facilitates a more accurate representation of the linear and nonlinear relationships between volume features and brain age. Results: Across various datasets, RFBLSO demonstrates the capability to formulate brain age prediction models, achieving a Mean Absolute Error (MAE) of 4.60 years within the Healthy Group and 4.98 years within the Chinese2020 dataset. In the Clinical Group, RFBLSO achieves measurement and effective differentiation among Healthy Controls (HC), Mild Cognitive Impairment (MCI), and Alzheimer's disease (AD) (MAE for HC, MCI, and AD: 4.46 years, 8.77 years, 13.67 years; the effect size ti2 of the analysis of variance for AD/MCI vs. HC is 0.23; the effect sizes of post-hoc tests are Cohen's d = 0.74 (AD vs. MCI), 1.50 (AD vs. HC), 0.77 (MCI vs. HC)). Compared to other linear or nonlinear brain age prediction methods, RFBLSO offers more accurate measurements and effectively distinguishes between Clinical Groups. This is because RFBLSO can simultaneously explore both linear and nonlinear relationships between brain region volume and brain age. Conclusion: The proposed RFBLSO effectively represents both linear and nonlinear relationships between brain region volume and brain age, allowing for more accurate individual brain age estimation. This provides a feasible method for predicting the risk of neurodegenerative diseases.