Objectives Several studies demonstrated a positive relationship between hemoglobin level and bone mineral density (BMD). Thus, the association between hemoglobin concentration and osteoporosis in elders with type 2 diabetes mellitus (T2DM) was explored in this study. Methods Totally, 573 elders with T2DM were included in the study. BMD was measured by dual-energy X-ray absorptiometry. Hemoglobin levels were tested. The association between the hemoglobin level and osteoporosis was subjected to logistic regression analysis. Results For men, the hemoglobin levels were significantly lower in osteoporosis group than that in non-osteoporosis group (135.98 +/- 16.20 vs. 142.84 +/- 13.78 g/L, P = 0.002). Hemoglobin levels were positively related with BMD of total hip and femoral neck in men (r = 0.170, P = 0.004; r = 0.148, P = 0.012, respectively). After adjusting for age, body mass index (BMI), hemoglobin A1c (HbA1c), estimated glomerular filtration rate (eGFR) and 25-hydroxyvitamin D-3 [25(OH) D-3], the hemoglobin level was related with a 0.97-fold lower risk of osteoporosis (odds ratio (OR): 0.97; 95% confidence interval (CI): 0.95-0.99; P = 0.004) in men, but no such association was found in women. Conclusion Higher levels of hemoglobin play a protective role against osteoporosis in older men with T2DM.