Lignin fibers typically influence the mixture performance of stone matrix asphalt (SMA), such as strength, stability, durability, noise level, rutting resistance, fatigue life, and water sensitivity. However, limited studies were conducted to analyze the influence of fibers on the percent voids in mineral aggregate in bituminous mixture (VMA) during the mixture design. This study analyzed the effect of different fibers and fiber contents on the VMA in SMA mixture design. A surface-dry condition method test and Marshall Stability test were applied on the SMA mixture with four different fibers (i.e., flocculent lignin fiber, mineral fiber, polyester fiber, blended fiber). The test results indicated that the bulk specific gravity of SMA mixtures and asphalt saturation decreased with the increasing fiber content, whilst the percent air voids in bituminous mixtures (VV), Marshall Stability and VMA increased. Mineral fiber had the most obvious impact on the bulk specific gravity of bituminous mixtures, while flocculent lignin fiber had a minimal impact. The mixture with mineral fiber and polyester fiber had significant effects on the volumetric properties, and, consequently, exhibited better VMA over the conventional SMA mixture with lignin fiber. Modified fiber content range was also provided, which will widen the utilization of mineral fiber and polyester fiber in the applications of SMA mixtures. The mixture evaluation suggested no statistically significant difference between lignin fiber and polyester fiber on the stability. The mineral fiber required a much larger fiber content to improve the mixture performance than other fibers. Overall, the results can be a reference to guide SMA mixture design.