Grazing exclusion has been proved to be one of the main measures for rehabilitating degraded arid steppes. However, the effect of this management practice on plant species diversity and composition is ambiguous, specially under prolonged droughts. Concurrently considering the responses of individual plant species, diversity of functional groups, α-diversity, and β-diversity (and its components) may be crucial to the holistic understanding of grazing exclusion effects on plant communities under drought conditions. Here, we investigated the response of these diversity measures to short-term sheep exclusion under severe drought episode in arid steppes of Alfa-grass (Stipa tenacissima) with a long evolutionary history of livestock grazing. Individual species responses were tested based on species occurrence and abundance in either grazed or grazing-excluded steppes, in addition, we used indicator species analysis to assess the strength of the association between plant species and management type. Likewise, α-diversity, abundance- and incidence-based β-diversity, as well as the functional groups’ diversities were quantified using Hill Numbers and compared between the two management types. Sheep grazing exclusion enabled the recovery of various Alfa-steppe indicator species and improved the size of regional species pool, overall α-diversity, and the diversity of therophytes. This management practice decreased the abundance-based β-diversity and the nestedness-resultant fraction of the incidence-based β-diversity at the local scale, while at the landscape scale increased the abundance-based β-diversity and its balanced variation fraction and reduced the incidence-based β-diversity and its turnover component. Furthermore, protection from grazing altered β-diversities scaling patterns by maintaining higher balanced variation in species abundance at large spatial scale and greater abundance-gradient in species composition at the fine-scale. Our results suggest that the implementation of short-term grazing exclusion in degraded arid steppes would be the appropriate management practice for vegetation restoration and plant diversity conservation during prolonged drought periods.