To solve large scale linear equations involved in the Fast Multipole Boundary Element Method (FM-BEM) efficiently, an iterative method named the generalized minimal residual method (GMRES)(m)algorithm with Variable Restart Parameter (VRP-GMRES(m) algorithm) is proposed. By properly changing a variable restart parameter for the GMRES(m) algorithm, the iteration stagnation problem resulting from improper selection of the parameter is resolved efficiently. Based on the framework of the VRP-GMRES(m) algorithm and the relevant properties of generalized inverse matrix, the projection of the error vector r_m+1 on r_m is deduced. The result proves that the proposed algorithm is not only rapidly convergent but also highly accurate. Numerical experiments further show that the new algorithm can significantly improve the computational efficiency and accuracy. Its superiorities will be much more remarkable when it is used to solve larger scale problems. Therefore, it has extensive prospects in the FM-BEM field and other scientific and engineering computing.