A graph based path planning method consisting of off-line part and on-line part for multi-UAV system in windy condition is proposed. In the off-line part, the task area is divided into grids. When two nodes is close enough, they are connected and weighted by the cost, obtained by solving an optimization problem based on difference method, between them. In order to ensure the accuracy of the difference method, the adjacent radius is set to be small enough. In order to ensure the generated paths close to analytical solution, the grid size is set to be much smaller than the adjacent radius. The shortest path algorithm is used to update the adjacent matrix and path matrix. In the on-line part, the target position assignment and optimal velocity can be obtained by accessing the adjacent matrix and path matrix. At the end of this paper, some numerical examples are taken to illustrate the validity of our method and the influence of the parameters.