A large number of studies have concluded that since housing pressure will affect the mobility of highly skilled young migrants (HSYM) in Chinese cities and regions, it is necessary to apply corresponding housing policies to adjust housing unaffordability for HYSM. This study uses a survey data conducted in China's Zhejiang Province of China, where specific policies have been implemented to attract talent and found that housing does crowds out the HSYM from a city, but the HSYM who have a master's degree or above or who work in government organizations or state-owned enterprises are more tolerant of housing unaffordability. The unmarried or those staying in the city for a long period are less tolerant of housing unaffordability. Meanwhile, there are the heterogeneous impacts of factors on the HSYM's tolerance for housing unaffordability across cities of different levels. Therefore, housing policies should highlight urban differences and intra-group differences.