Context: Systematic evaluation of a practitioner’s adherence and competence in delivering psychotherapeutic interventions can be complex. This study describes the development of a fidelity assessment tool for the Reitman Centre CARERS Program (RCCP), a carer group psychotherapy intervention with multiple didactic and clinical components. The tool’s value in informing psychotherapy training and best practice for practitioners from diverse professional settings is examined. Methods: The RCCP Fidelity Assessment Tool (RCCP-FAT) was developed following an iterative process of item writing and checking. Seven components of the RCCP – Group Structure, Dementia Education, Problem Solving Techniques, Therapeutic Simulation, Vertical/Horizontal Cohesion, Overall Global Rating – were assessed, with 3 to 8 items and a “global score” assigned to each. Fifteen trained raters were paired-up to rate 36 RCCP sessions using RCCP-FAT. Rater agreement, correlation between itemized and global scores, and correlation between global ratings and RCCP participants’ satisfaction were calculated. Results: 1188 RCCP-FAT items were rated by each of the 2 Rater Groups. Rater agreement was calculated to be 54.3% (κ = 0.32; 95% CI, 0.02681- 0.3729). A positive correlation was found between itemized and global scoring for four RCCP components evaluated (R = 0.833 to 0.929; p <0.01). Global score and participants’ satisfaction with “Simulation” was also positively correlated (R = 0.626, p <0.01). Conclusions: The study provided evidence for fair rater agreement for all RCCP-FAT assessment items. More importantly, the process of developing the tool systematically crystallized the clinical elements of RCCP, and helped to standardize training methods by creating a framework for providing feedback to learners that match the items on the RCCP-FAT. The use of RCCP-FAT to guide training and mentoring of incoming group leaders is essential in the scaling and dissemination of a complex training method like the RCCP to ensure fidelity to the original evidence-based intervention.Context: Systematic evaluation of a practitioner’s adherence and competence in delivering psychotherapeutic interventions can be complex. This study describes the development of a fidelity assessment tool for the Reitman Centre CARERS Program (RCCP), a carer group psychotherapy intervention with multiple didactic and clinical components. The tool’s value in informing psychotherapy training and best practice for practitioners from diverse professional settings is examined. Methods: The RCCP Fidelity Assessment Tool (RCCP-FAT) was developed following an iterative process of item writing and checking. Seven components of the RCCP – Group Structure, Dementia Education, Problem Solving Techniques, Therapeutic Simulation, Vertical/Horizontal Cohesion, Overall Global Rating – were assessed, with 3 to 8 items and a “global score” assigned to each. Fifteen trained raters were paired-up to rate 36 RCCP sessions using RCCP-FAT. Rater agreement, correlation between itemized and global scores, and correlation between global ratings and RCCP participants’ satisfaction were calculated. Results: 1188 RCCP-FAT items were rated by each of the 2 Rater Groups. Rater agreement was calculated to be 54.3% (κ = 0.32; 95% CI, 0.02681- 0.3729). A positive correlation was found between itemized and global scoring for four RCCP components evaluated (R = 0.833 to 0.929; p <0.01). Global score and participants’ satisfaction with “Simulation” was also positively correlated (R = 0.626, p <0.01). Conclusions: The study provided evidence for fair rater agreement for all RCCP-FAT assessment items. More importantly, the process of developing the tool systematically crystallized the clinical elements of RCCP, and helped to standardize training methods by creating a framework for providing feedback to learners that match the items on the RCCP-FAT. The use of RCCP-FAT to guide training and mentoring of incoming group leaders is essential in the scaling and dissemination of a complex training method like the RCCP to ensure fidelity to the original evidence-based intervention.