In this study, we explore the use of 3D convolutional neural networks (CNNs) for predicting internal temperature fields from the surface temperature, with a focus on applications where small temperature gradients, similar to those in the human body, are present. The network accuracy was evaluated under both ideal and non-ideal conditions which include noise and background temperature effects. In non-ideal scenarios, the network accurately reconstructed the 3D temperature field for small phantoms (e.g., 10 cm in diameter). However, as the size of the domain increased, the network’s predictive capacity diminished, particularly in regions far from the surface. To address this limitation, we introduced statistical uncertainty during training, simulating non-ideal conditions, in combination with a physics-informed loss function which embed the heat equation directly into the training process. This combination can improve the model’s performance, particularly in noisy environments, where traditional CNN architectures failed to reconstruct hot-spots in deeper regions. Our results suggest that combining deep learning with physical constraints offers a robust framework for non-invasive thermal imaging and other applications requiring high-precision temperature field reconstruction.