The group delay response of chirped fiber Bragg gratings (CFBG) written in polarization-maintaining fiber can be measured using the beat spectrum of the reflections from the two polarization axis, with a resolution and reproducibility better than 50 fs. The method is theoretically described, and we analyze the effect of group delay ripples (GDR) on the performance of a stretcher-compressor pair used in chirped pulse amplification systems. Ripples with a spectral period of the order of the pulse bandwidth are the most deleterious. Those are readily captured with our method, despite the limit on resolution due to the fiber polarization beat length. As a rule of thumb, the GDR should not exceed half the pulse duration. The reconstructed phase of the CFBG is also used to predict the pulse compression performance for a given device. Measurement of GDR as low as 161 fs is experimentally demonstrated.