The enzyme-linked immunosorbent assay (ELISA) coupled with surface-enhanced Raman spectroscopy (SERS) detection is a growing area of analytical chemistry emerged as a solution to a problem of sensitivity improvement. SERS-based ELISA with horseradish peroxidase as an enzymatic label, o-phenylenediamine (oPD) substrate, and 2,3-diaminophenazine (DAP) enzymatic product was one of the first examples of such systems. Surprisingly, the full potential of this long-known approach has not been revealed yet. The current study addresses a previously neglected problem of SERS detection stage performance. Using silver nanoparticles and model mixtures of oPD and DAP, the effects of pH, concentration of aggregating agent, and particle surface chloride stabilizer were extensively evaluated. At optimal mildly acidic pH of 3, 0.93 to 1 M citrate buffer, and AgNPs stabilized with 20 mM chloride, at least two orders of magnitude advantages in the detection limits of SERS over colorimetry both for DAP and HRP were demonstrated. We suppose that this improved detection system could become a useful tool for the development of SERS-based ELISA protocols.