Photovoltaic power generation has high variability and uncertainty because it is affected by uncertain factors such as weather conditions. Therefore, probabilistic forecasting is useful for optimal operation and risk hedging in power systems with large amounts of photovoltaic power generation. However, deterministic forecasting is the mainstay of photovoltaic generation forecasting; there are few studies on probabilistic forecasting and feature selection from weather or time-oriented features in such forecasting. In this study, prediction intervals were generated by the lower upper bound estimation using neural networks with two outputs to make probabilistic predictions. The objective was to improve prediction interval coverage probability (PICP), mean prediction interval width (MPIW), and loss, which is the integration of these two metrics, by removing unnecessary features through feature selection. When features with high gain were selected by random forests (RF), in the forecast of 14.7-kW PV systems, loss improved by 1.57 kW, PICP by 0.057, and MPIW by 0.12 kW on average over two weeks compared to the case where all features were used without feature selection. Therefore, the low gain features from RF act as noise in LUBE and reduce the prediction accuracy.