For multi-sensor integrated systems, such as a mobile mapping system (MMS), data fusion at sensor-level, i.e., the 2D-3D registration between optical camera and LiDAR, is a prerequisite for higher level fusion and further applications. This paper proposes a line-based registration method for panoramic images and LiDAR point cloud collected by a MMS. We first introduce the system configuration and specification, including the coordinate systems of the MMS, the 3D LiDAR scanners, and the two panoramic camera models. We then establish the line-based transformation model for panoramic camera. Finally, the proposed registration method is evaluated for two types of camera models by visual inspection and quantitative comparison. The results demonstrate that the line-based registration method can significantly improve the alignment of the panoramic image and LiDAR datasets under either the ideal spherical or the rigorous panoramic camera model, though the latter is more reliable.