Human Face and facial parts are the most significant parts as it reveals a person’s true identity. It plays an important role in various biometric applications like crowd analysis, human tracking, photography, cosmetic surgery, etc. There are many techniques are available to detect a facial image. Among them, skin detection is the most popular one. The aim of this paper is to detect first the person's identity from facial image and finally check any spot present the the detected person. The first step is to detect the maximum skin region based on a combination method of RGB and HSV color space model. Next it is to verify the skin areas of human through machine learning approach. The Aggregated Channel Features (ACF) detector is used to identify the different facial parts like eye pairs, nose, and mouth. Bootstrap aggregation decision tree classifier is applied to classify the person’s identity based on Histogram Oriented Gradient (HOG) features value. The experimental results show that the proposed method gives the average 97% accuracy.