Environmental exposure to heavy metals is suspected to result in neuropathology damage and cognitive impairment. We aimed to explore the association of Alzheimer's disease (AD) risk with the internal dose of heavy metals by constructing a hospital-based case-control study and using propensity-score-matching methods. We investigated 170 patients with AD and 264 controls from the Department of Neurology and Family Medicine, China Medical University Hospital in Taiwan. All patients with AD received clinical neuropsychological examination and cognitive-function assessments, including the mini-mental status examination and clinical dementia rating scale. We also constructed a propensity-score-matched population of 82 patients with AD and 82 controls by matching age, gender, education, and AD-related comorbidity. Blood levels with cadmium, lead, mercury, selenium, and urinary arsenic profile were measured. Logistic regression models and 95% confidence intervals (CIs) were applied to estimate AD risk. After stratification by respective quartile cutoffs of heavy metals, the AD risk of study participants with high urinary inorganic arsenic (InAs%) or low dimethylarsinic acid (DMA%) significantly increased (p < 0.05), as similarly found in the propensity-score-matched population. In addition, people with a low median level of selenium and high median level of InAs%, or/and a low median level of DMA% had approximately two- to threefold significant AD risk. Urinary arsenic profiles may be associated with increased AD risk. Repeat measurements of heavy metals with large sample size and the surveying of potential exposure sources are recommended in future studies.