Perennial cropping play vital roles in regulating soil carbon sequestration and thus mitigating climate change. However, how perennial cropping affects soil microbial community remains elusive. Using a field investigation, this study was conducted to examine the effects of mugwort cropping along a chronosequence (that is, wheat-maize rotation, 3-year, 6-year, and 20-year mugwort cropping) on soil microbial community in temperate regions of Northern China. The results showed that the highest total, actinomycetes, and fungi PLFAs were found in the 3-year mugwort cropping soils. All PLFAs of microbial groups were lowest in the 20-year mugwort cropping soils. All of the three cropping years of mugwort increased network complexity of soil microbial community. Changes in total nitrogen and phosphorus content as well as the ratio of ammonium nitrogen to nitrate nitrogen could be primarily explain the variations in soil microbial community along the mugwort cropping chronosequence. Our observations highlight the contrasting impacts of soil microbial community to short-term and long-term mugwort cropping compared to conventional rotations and would have critical implications for sustainable agricultural management under perennial cropping in temperate regions.