In recent years, many algorithms based on end-to-end deep networks have been proposed to deal with the target detection problem of videos. However, the deep network models usually consume a lot of computing resources during the procedure of analysis of videos with complex dynamic backgrounds. In this paper, a new method of object detection based on information theory is presented. Firstly, each frame in a video is converted into an effective information map by using the Harris corner detection method. Secondly, the sensitive areas in the frame are extracted by using the context information and the effective information maps of the consecutive video frames. The sensitive areas in the video frame are the candidate areas where the target objects would be appeared at high probabilities. Thirdly, the information entropy features of each sensitive area are extracted to form the feature matrix, based on which, an SVM model is trained for selecting the target areas from the sensitive areas. Finally, the locations of the objects are detected based on the target areas in the video with a complex dynamic background. As a lightweight video detection framework, the method presented in this paper can save a lot of computing resources. Experimental results show that this method can achieve good results in the benchmark of CDnet 2014.