The rapid growth of smartphone financial services raises the need for secure mobile authentication. Continuous authentication is a user-friendly way to strengthen the security of smartphones by implicitly monitoring a user’s identity through sessions. Mobile continuous authentication can be viewed as an anomaly detection problem in which models discriminate between one genuine user and the rest of the imposters (anomalies). In practice, complete imposter profiles are hardly available due to the time and monetary cost, while leveraging genuine data alone yields poor generalized models due to the lack of knowledge about imposters. To address this challenge, we recast continuous mobile authentication as a few-shot anomaly detection problem, aiming to enhance the inference robustness of unseen imposters by using partial knowledge of available imposter profiles. Specifically, we propose a novel deep learning-based model, namely Local Feature Pooling based Temporal Convolution Network (LFP-TCN), which directly models raw sequential mobile data, aggregating global and local feature information. In addition, we introduce a random pattern mixing augmentation to generate class-unconstrained imposter data for the training. The augmented pool enables characterizing various imposter patterns from limited imposter data. Finally, we demonstrate practical continuous authentication using score-level fusion, which prevents long-term dependency or increased model complexity due to extended re-authentication time. Experiments on two public benchmark datasets show the effectiveness of our method and its state-of-the-art performance.