Connected and Autonomous Vehicle (CAV) combines technologies of autonomous vehicle (AV) and connected vehicles (CV) to develop quicker, more reliable and safer traffic. Artificial Intelligence (AI) based CAV solutions play significant roles in sustainable city. The convergence imposes stringent security requirements for CAV safety and reliability. In practice, vehicles are developed with increased automation and connectivity. Increased automation increases the reliance on the sensor-based technologies and decreases the reliance on driver; increased connectivity increases the exposures of vehicles vulnerability and increases the risk for an adversary to implement a cyber-attack. Much work has been dedicated to identifying the security vulnerabilities and recommending mitigation techniques associated with different sensors, controllers, and connection mechanisms, respectively. However, there is an absence of comprehensive and in-depth studies to identify how the cyber-attacks exploit the vehicles vulnerabilities to negatively impact the performance and operations of CAV. In this survey, we set out to thoroughly review the security issues introduced by AV and CV technologies, analyze how the cyber-attacks impact the performance of CAV, and summarize the solutions correspondingly. The impact of cyber-attacks on the performance of CAV is elaborated from both viewpoints of intra-vehicle system and inter-vehicle system. We pointed out that securing the perception and operations of CAV would be the top requirement to enable CAV to be applied safely and reliably in practice. Also, we suggested to utilize cloud and new AI methods to defend against smart cyber-attacks on CAV.