Cognitive impairment and sarcopenia may share common risk factors and pathophysiological pathways. This study was performed to examine the association between impairments in specific cognitive domains and sarcopenia (and its defining components) in a large group of community-dwelling older adults. Cross-sectional analysis was performed on the baseline data of 3,014 adults aged 70–84 years enrolled in the Korean Frailty and Aging Cohort Study (KFACS). The final analysis included 1,887 adults underwent dual-energy X-ray absorptiometry and cognitive function assessments. Those with disability in activities of daily living, dementia, severe cognitive impairment, Parkinson’s disease, musculoskeletal complaints, neurological disorders, or who were illiterate were excluded. Cognitive function was assessed using the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet, the Frontal Assessment Battery. For sarcopenia, we used the diagnostic criteria of the Asian Working Group for Sarcopenia. The prevalence of sarcopenia was 9.6% for men and 7.6% for women. Sarcopenia (odds ratio [OR] 1.76, 95% confidence interval [CI] 1.04–2.99) and slow gait speed (OR 2.58, 95% CI 1.34–4.99) were associated with cognitive impairment in men. Only slow gait speed (OR 1.88, 95% CI 1.05–3.36) was associated with cognitive impairment in women. Sarcopenia is associated with cognitive impairment mainly due to slow gait speed. Our results suggested that cognitive impairment domains, such as processing speed and executive function, are associated with sarcopenia-related slow gait speed.