Background Older subjects have a higher risk for vertebral compression fracture. Maintaining a higher bone mineral density (BMD) at this age can protect individuals from osteoporosis-related events. Body mass index (BMI) has been found to have a robust association with BMD. However, excessive BMI is detrimental to bone health and may cause systemic disorders. Therefore, the present study aimed to determine the association between BMI and BMD, and identify a reasonable BMI range. Methods A total of 961 participants were recruited from community-dwelling residents between August 2021 and May 2022. A weighted multivariate linear regression model was applied to identify the relationship between BMI and BMD. Meanwhile, subgroup stratified analysis by BMI quartile and gender was also performed. A non-linear relationship and threshold value were determined based on the smooth curve fittings and threshold effects analysis model. Results A robust relationship was found between BMI and BMD, which remained significant in subgroups stratified by gender and BMI quartile. The BMI inflection point values in lumbar BMD and femoral neck BMD were 25.2 kg/m(2) and 27.3 kg/m(2), respectively. For individuals with BMI < 25.2 kg/m(2), an increase in BMI was related to an increase in lumbar BMD. For BMI > 25.2 kg/m(2), an increase in BMI was associated with a decrease in lumbar BMD. For subjects with BMI < 27.3 kg/m(2), the femoral neck BMD rose by 0.008 kg/m(2) for each unit rise in BMI. However, when BMI exceeded 27.3 kg/m(2), the femoral neck BMD increased only by 0.005 kg/m(2). Fracture risk assessment based on the spinal deformity index (SDI) failed to determine the optimal BMI range. Conclusions This study found an inflection point between BMI and lumbar/ femoral neck BMD in older community-dwelling subjects. An appropriate BMI but not an excessive BMI may allow older adults to have a better BMD.