Besides the intensity and wavelength, the ability to analyze the optical polarization of the detected light can provide a new degree of freedom for numerous applications, such as object recognition, biomedical applications, environmental monitoring and remote sensing imaging. However, the conventional filter-integrated polarimetric sensing systems require complex optical components and complicated fabrication process, severely limiting its on-chip miniaturization and functionalities. Herein, the reconfigurable polarimetric photodetection with photovoltaic mode are developed based on a few-layer MoS2/PdSe2 heterostructure channel and a charge-trap structure composed of Al2O3/HfO2/Al2O3 (AHA) stacked dielectric. Because of the remarkable charge-trapping ability of carriers in AHA stack, the MoS2/PdSe2 channel exhibits a high program/erase current ratio of 105 and memory window exceeding 20 V. Moreover, photovoltaic mode of MoS2/PdSe2 Schottky diode can be operated and manipulable, resulting into high and distinct responsivities in the visible broadband. Interestingly, the linear polarization of device can be modulated under program/erase states, enabling the reconfigurable capability of linearly polarized photodetection. This study demonstrates a new prototype heterostructure-based photodetector with capability of both tunable responsivity and linear polarization, demonstrating great potential application toward reconfigurable photosensing and polarization-resolved imaging applications.