Purpose This cross-sectional study aims to explore the association between frailty and domains of quality of life (QOL) among hospitalized patients in older age under consideration of the sociodemographic characteristics. Methods Older inpatients in a comprehensive tertiary hospital in China were involved. Data including sociodemographic, clinical diagnosis, frailty status, and QOL were collected through questionnaires and physical assessments. Frailty was measured using the frailty phenotype defined by Fried. QOL was determined by completion of version 2 of the 12-item Short Form Survey (SF-12v2). Multivariable linear regression and structural equation modeling were applied to estimate associations between frailty and QOL. Results 260 older inpatients were enrolled and classified as frail (31.92%), pre-frail (39.62%), and robust (28.46%). Patients with frailty were older and were more likely to have severe comorbidity, higher nutritional risk score and lower body mass index, activities of daily living score (P < 0.05). As the extent of frailty increased, the patients in old age experienced poor QOL in most of dimensions. Five multiple linear regression models indicated that the estimated differences in the physical component summary of QOL were lower for the frail and pre-frail groups versus the robust group. After adjusting all covariates, the coefficients were - 7.72 (- 11.04, - 4.41) and - 16.48 (- 20.37, - 12.58) for pre-frail and frail, respectively. Structural equation modeling results showed that sociodemographic characteristics directly affected QOL, while others played an indirect role in reducing their QOL by causing frailty. Conclusions Frailty is prevalent in hospitalized patients having older age. Patients with frailty would experience poor QOL, especially in the dimension of physical health. To promote healthy aging, further studies into how frailty influences the QOL is warranted.