Immunohistochemistry is a widely used technique for research and diagnostic purposes that relies on the recognition by antibodies of antigens expressed in tissues. However, tissue processing and particularly formalin fixation affect the conformation of these antigens through the formation of methylene bridges. Although antigen retrieval techniques can partially restore antigen immunoreactivity, it is difficult to identify antibodies that can recognize their target especially in formalin-fixed paraffin-embedded tissues. Most of the antibodies currently used in immunohistochemistry have been obtained by animal immunization; however, in vitro display techniques represent alternative strategies that have not been fully explored yet. This review provides an overview of phage display-based antibody selections using naïve antibody libraries on various supports (fixed cells, dissociated tissues, tissue fragments, and tissue sections) that have led to the identification of antibodies suitable for immunohistochemistry.