Despite the ubiquity of the Boston naming test (BNT) in clinical practice and research, concerns have been expressed about its poor quality pictures, insufficient psychometric properties, and cultural bias in non-English language backgrounds. We modified the black-and-white BNT with a set of color pictures since color effects have been suggested to improve naming accuracy in the visual naming test. This study aimed to examine and compare the reliability and validity of the color-picture version of BNT (CP-BNT) and the black-and-white version of BNT (BW-BNT) to differentiate amnestic mild cognitive impairment (aMCI) or mild Alzheimer's disease (AD) from the cognitive normals. This study included two subgroups, and each subgroup had 101 normal controls, 51 aMCI, and 52 mild AD. One subgroup undertook BW-BNT and the other conducted CP-BNT. The reliability, convergent and discriminant validity, and the diagnostic accuracy of two versions of BNT were evaluated. The CP-BNT showed a greater area under the curve (AUC) than the BW-BNT for aMCI (80.3 vs.s 69.4%) and mild AD (93.5 vs. 77.6%). The CP-BNT also demonstrated better convergent validity with CDR global scores and better reliability (Cronbach's coefficient 0.66 for the CP-BNT vs. 0.55 for the BW-BNT). At the optimal cutoff value of spontaneous naming, the CP-BNT demonstrated improved sensitivity and specificity for differentiating mild AD from NC with a higher positive predictive value, negative predictive value, and lower false-positive rate. Compared with BW-BNT, CP-BNT is a more reliable and valid test to assess cognitive and naming impairment.Copyright © 2022 Li, Yu, Hu, Zhang, Liu, Fan, Ruan and Wang.