Over the past decade, our knowledge of the $\gamma$-ray sky has been revolutionized by ground- and space-based observatories by detecting photons up to several hundreds of tera-electron volt (TeV) energies. A major population of the $\gamma$-ray bright objects are active galactic nuclei (AGN) with their relativistic jets pointed along our line-of-sight. Gamma-ray emission is also detected from nearby mis-aligned AGN such as radio galaxies. While the TeV-detected radio galaxies ($TeVRad$) only form a small fraction of the $\gamma$-ray detected AGN, their multi-wavelength study offers a unique opportunity to probe and pinpoint the high-energy emission processes and sites. Even in the absence of substantial Doppler beaming $TeVRad$ are extremely bright objects in the TeV sky (luminosities detected up to $10^{45}~erg~s^{-1}$), and exhibit flux variations on timescales shorter than the event-horizon scales (flux doubling timescale less than 5 minutes). Thanks to the recent advancement in the imaging capabilities of high-resolution radio interferometry (millimeter very long baseline interferometry, mm-VLBI), one can probe the scales down to less than 10 gravitational radii in $TeVRad$, making it possible not only to test jet launching models but also to pinpoint the high-energy emission sites and to unravel the emission mechanisms. This review provides an overview of the high-energy observations of $TeVRad$ with a focus on the emitting sites and radiation processes. Some recent approaches in simulations are also sketched. Observations by the near-future facilities like Cherenkov Telescope Array, short millimeter-VLBI, and high-energy polarimetry instruments will be crucial for discriminating the competing high-energy emission models.