Background: Severe cerebrovascular events are associated with carotid atherosclerotic plaque progression and rupture that is mediated by inflammation. F-18-fluorodeoxyglucose ([F-18]FDG) PET is important for assessing the inflammation of carotid atherosclerotic plaque, but it suffers from the limitations of radiation exposure. Additionally, inflammation of perivascular adipose tissue (PVAT) has been found to promote atherosclerosis progression through paracrine signaling mechanisms. The study aimed to develop an ensemble model based on carotid plaque and PVAT MRI radiomics for identifying highly inflammatory plaques (HIPs). Results: 159 asymptomatic carotid atherosclerosis patients (137 males; 65 +/- 8 years old) with 209 plaques (104 HIPs) were consecutively enrolled. 47.95% (70/146) of cases and 53.97% (34/63) were defined as HIPs in the training and testing datasets, respectively. There was more lipid core, more intraplaque hemorrhage, and less calcification in the HIPs compared to the non-highly inflammatory plaques (NHIPs) in the training dataset (p = 0.002, 0.019, and 0.013, respectively). Notably, the incidence of indistinct PVAT (IPVAT) in HIPs was higher than that in NHIPs, both in the training (81.43% vs. 46.05%; p < 0.001) and the testing (88.24% vs. 58.62%; p = 0.007) datasets. The correlations between plaque MRI characteristics and [F-18]FDG uptake differed between the NHIPs and HIPs. However, IPVAT consistently correlated with SUVmax (r = 0.35, 0.30; p < 0.001, p = 0.002; for NHIPs and HIPs, respectively). The ensemble model that integrates the radiomics of carotid plaque and PVAT outperformed all models in predicting HIP (area under the curve [AUC] = 0.92/0.91, training/testing dataset). The follow-up further validated the PET for predicting plaque progression with the same accuracy as the ensemble model (AUC: 0.85 vs. 0.79). Conclusions: The ensemble model integrating the radiomics of carotid plaque and perivascular adipose tissue provides an equivalent tool to PET in the visualization of the evaluation of carotid atherosclerosis inflammation and progression.