The present work deals with the development of finite element methodology for obtaining the stress distributions in thick cylindrical HK40 stainless steel pipe that carry high temperature fluids. The material properties and loading are assumed to be random variables. Thermal stresses that are generated along radial, axial and tangential directions are computed generally using analytical expressions which are very complex. To circumvent such an issue, the probability theory and mathematical statistics have been applied to many engineering problems which allows to determine the safety both quantitatively and objectively based on the concepts of reliability. Monte Carlo simulation methodology is used to study the probabilistic characteristics of thermal stresses which is used for estimating the probabilistic distributions of stresses against the variations arising due to material properties and load. A 2-D Probabilistic finite element code is developed in MATLAB and the deterministic solution is compared with ABAQUS solutions. The values of stresses that are obtained from the variation of elastic modulus are found to be low as compared to the case where the load alone is varying. The probability of failure of the pipe structure is predicted against the variations in internal pressure and thermal gradient. These finite element framework developments are useful for the life estimation of piping structures in high temperature applications and subsequently quantifying the uncertainties in loading and material properties.