Fusarium head blight (FHB) disease is extensively distributed worldwide. This disease damages grain quality and reduces yield. The detection of this disease in a high throughput way is crucial to planters and breeders. Our study focused on developing a method for processing wheat color images and accurately detecting disease areas using deep learning and image processing techniques. The color images of wheat at the milky stage were collected and processed to construct datasets, which were used to retrain a deep convolutional neural network model using transfer learning. Testing results showed that the model can detect spikes, and the coefficient of determination of the number of spikes between the manual count and the detection was 0.80. The model was assessed, and the mean average precision for the testing dataset was 0.9201. On the basis of the results of spike detection, a new color feature was applied to obtain the gray image of each spike. Then, a modified region growing algorithm was implemented to segment and detect the diseased areas of each spike. Results show that the region growing algorithm performs better than K-means and Otsu’s method in segmenting the FHB disease. Overall, this study demonstrates that deep learning techniques enable the accurate detection of FHB in wheat using color images, and the proposed method can effectively detect spikes and diseased areas, thereby improving the efficiency of FHB detection.