Gesture recognition technology has been quickly developed in the field of human-computer interaction. The multiple-input multiple-output (MIMO) radar has been widely adopted in gesture recognition because of its notable spatial resolution. In this work, a highly accurate MIMO radar-based hand gesture recognition algorithm with very low complexity is proposed. To make the proposed system applicable in the industry and work well even when the size of the training data is limited, we applied several low-complexity adaptive signal processing methods to extract features and reduce the noise effect. First, spectrum analysis is applied to range-Doppler maps (RDMs) and a cell-averaging constant false alarm rate (CA-CFAR) with mirror filters is applied to improve the robustness to noise. Afterward, the features related to the distance, speed, direction, and the elevation angle of the moving object are determined by the proposed adaptive signal analysis techniques and the random forest is applied for classification. The proposed system has the capability to precisely distinguish and identify eight motions, including waving, moving to the left or right, patting, pushing, pulling, and rotating clockwise or anticlockwise, with an accuracy of 95%. Experiments demonstrate the capability of the proposed hand gesture recognition system to classify different movements precisely.