preprints.org > computer science and mathematics > artificial intelligence and machine learning > doi: 10.20944/preprints202408.1892.v1 (registering DOI)

Preprint Article Version 1 This version is not peer-reviewed

Version 1 : Received: 25 August 2024 / Approved: 26 August 2024 / Online: 27 August 2024 (10:25:48 CEST)

Recently, a large number of Transformer-based solutions have emerged for the trajectory prediction task, but there are shortcomings in the effectiveness of Transformers in trajectory prediction. Specifically, while position encoding preserves some of the ordering information, the self-attention mechanism at the core of the Transformer has its alignment invariance that leads to the loss of temporal information, which is crucial for trajectory prediction. For this reason, we design a simple and efficient strategy for temporal information extraction and prediction of trajectory sequences using the self-attention mechanism and linear layers. Experimental results show that the strategy we designed can effectively combine the advantages of the linear layer and self-attention mechanisms, compensating for the shortcomings of the Transformer. Additionally, we conducted an empirical study to explore the effectiveness of the linear layer and sparse self-attention mechanisms in trajectory prediction.

Trajectory Prediction; Linear; self-Attention; Autonomous Driving

Computer Science and Mathematics, Artificial Intelligence and Machine Learning

* All users must log in before leaving a comment