The International Roughness Index (IRI) is the standard scale for evaluating road roughness in many countries in the world. The Taipei City government actively promotes a Road Smoothing Project and plans to complete the rehabilitation of the main and minor roads within its jurisdiction. This study aims to detect the road surface roughness in Taipei City and recommend appropriate IRI thresholds for road rehabilitation. A total of 171 asphalt concrete pavement sections in Taipei City with a total length of 803.49 km were analyzed and compared by IRI. The longitudinal profile of the detected road sections was measured using an inertial profiler. The statistical analysis showed that the IRI value prior to road leveling was mainly distributed between 5 and 8 m/km, while the IRI value after road leveling was mainly distributed between 3 and 4.5 m/km. This confirms that the implementation of the Road Smoothing Project has a significant effect on improving road smoothness. Moreover, based on the analysis results, it is recommended that the IRI threshold value for road rehabilitation in Taipei City be set at 4.50 m/km.