Clustering white matter (WM) tracts from diffusion tensor imaging (DTI) is primarily important for quantitative analysis on pediatric brain development. A recently developed algorithm, density peaks (DP) clustering, demonstrates great robustness to the complex structural variations of WM tracts without any prior templates. Nevertheless, the calculation of densities, the core step of DP, is time consuming especially when the number of WM fibers is huge. In this paper, we propose a fast algorithm that accelerates the density computation about 50 times over the original one. We convert the global calculation for the density as well as critical parameter in the process into local computations, and develop a binary tree structure to orderly store the neighbors for these local computations. Hence, the density computation turns out to be a direct access of the structure, rendering significantly computational saving. Performing experiments on synthetic point data and the JHU-DTI data set and comparing results of our fast DP algorithm and existing clustering methods, we can validate the efficiency and effectiveness of our fast DP algorithm. Finally, we demonstrate the application of the proposed algorithm on the analysis of pediatric WM tract development.