Article
Version 1
This version is not peer-reviewed
Attention-Linear Trajectory Prediction
Version 1
: Received: 25 August 2024 / Approved: 26 August 2024 / Online: 27 August 2024 (10:25:48 CEST)
How to cite: Wang, B.; He, L.; Song, L.; Niu, R.; Cheng, M. Attention-Linear Trajectory Prediction. Preprints 2024, 2024081892. https://doi.org/10.20944/preprints202408.1892.v1 Wang, B.; He, L.; Song, L.; Niu, R.; Cheng, M. Attention-Linear Trajectory Prediction. Preprints 2024, 2024081892. https://doi.org/10.20944/preprints202408.1892.v1
Abstract
Recently, a large number of Transformer-based solutions have emerged for the trajectory prediction task, but there are shortcomings in the effectiveness of Transformers in trajectory prediction. Specifically, while position encoding preserves some of the ordering information, the self-attention mechanism at the core of the Transformer has its alignment invariance that leads to the loss of temporal information, which is crucial for trajectory prediction. For this reason, we design a simple and efficient strategy for temporal information extraction and prediction of trajectory sequences using the self-attention mechanism and linear layers. Experimental results show that the strategy we designed can effectively combine the advantages of the linear layer and self-attention mechanisms, compensating for the shortcomings of the Transformer. Additionally, we conducted an empirical study to explore the effectiveness of the linear layer and sparse self-attention mechanisms in trajectory prediction.
Keywords
Trajectory Prediction; Linear; self-Attention; Autonomous Driving
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (0)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment