This paper presents an Edge-based color image segmentation derived from the method of Particle Motion in a Vector Image Fields (PMVIF) that could previously be applied only to monochrome images. Instead of using an edge vector field derived from a gradient vector field and a normal compressive vector field derived from a Laplacian-gradient vector field, two novel orthogonal vector fields, directly computed from a color image, one parallel and another orthogonal to the edges, were used in the model to force a particle to move along the object edges. The normal compressive vector field is derived from the center-to-centroid vectors of local color distance images. Next, the edge vector field is derived by taking the normal compressive vector field, multiplied by differences of auxiliary image pixels to obtain a vector field analogous to a Hamiltonian gradient vector field. Using the PASCAL Visual Object Classes Challenge 2012 (VOC2012) and the Berkeley Segmentation Data Set and Benchmarks 500 (BSDS500), the benchmark score of the proposed method is provided in comparison with those of the traditional PMVIF, Watershed, SLIC, K-means, Mean shift, and JSEG. The proposed method yields better RI, GCE, NVI, BDE, Dice coefficients, faster computation time, and noise resistance.