Background: Previously reported superior tumor response of pemetrexed in the second-line setting for advanced non-squamous non-small cell lung cancer (advNS-NSCLC) has never been confirmed in real-world studies. Platinum-based doublet is frequently used in the second-line setting for advanced NSCLC in China. Methods: A retrospective cohort study was conducted including patients receiving pemetrexed or docetaxel-based chemotherapy in the second-line setting for advNS-NSCLC in four Chinese tertiary care hospitals. Propensity score matched treatment groups were created for head-to-head comparisons on best tumor response and clinical toxicity. Multiple regression analyses were performed to rank the impact of the four regimens on the risks of tumor progression and hematological adverse events. Results: Three hundred and eighty-four patients were included for creating matched treatment groups for pemetrexed versus platinum/pemetrexed (33 pairs), docetaxel (17 pairs), and platinum/docetaxel (29 pairs), respectively. No significant differences were identified for best tumor response between pemetrexed and the other three regimens. However, pemetrexed was associated with significantly fewer patients experiencing anemia (39.4% vs. 69.7%, P = 0.004) and neutropenia (6.1% vs. 30.3%, P = 0.021) than platinum/pemetrexed. Multiple regression analyses indicated that pemetrexed was associated with significantly slower tumor progression (hazard ratio 0.628, P = 0.040) and a significantly lower risk of neutropenia (odds ratio 0.132, P = 0.019) than docetaxel. Conclusions: Pemetrexed was associated with significantly postponed tumor progression and significantly less hematological toxicity than docetaxel in the real-world second-line setting for advNS-NSCLC in Chinese patients. Pemetrexed monotherapy had comparable tumor response, but significantly less hematological toxicity than pemetrexed-based doublet.