The interest on amorphous carbon thin films with low secondary electron yield (SEY) has been increasing in the last years, to mitigate electron multipacting in particle accelerators and in RF devices. Previous works found that the SEY increases with the hydrogen amount and correlates with the Tauc gap. In this work, we analyse films produced by magnetron sputtering with different contents of hydrogen and deuterium, incorporated via target poisoning and sputtering of CxDy molecules. XPS was implemented to estimate the phase composition of the films. The maximal SEY was found to decrease linearly with the fraction of the graphitic phase in the films. These results are supported by Raman scattering and UPS. The graphitic phase decreases almost linearly for hydrogen and deuterium concentrations between 12% and 46% (at.), but abruptly decreases when the concentration reaches 53%. This vanishing of the graphitic phase is accompanied by a strong increase of SEY and of the Tauc gap. These results suggest that the SEY is not dictated directly by the concentration of H/D but by the fraction of graphitic phase. The results are supported by a model used to calculate SEY of films consisting of a mixture of graphitic and polymeric phases.