In the paper, the authors establish two identities to express the generating function of the Chebyshev polynomials of the second kind and its higher order derivatives in terms of the generating function and its derivatives each other, deduce an explicit formula and an identities for the Chebyshev polynomials of the second kind, derive the inverse of an integer, unit, and lower triangular matrix, present several identities of the Catalan numbers, and give some remarks on the closely related results including connections of the Catalan numbers respectively with the Chebyshev polynomials, the central Delannoy numbers, and the Fibonacci polynomials.