1. Introduction
Audit&feedback (A&F) is a proven and widely used methodology for improving continuous quality of health care. It is essentially based on two aspects: audit, a systematic review of the quality of processes and outcomes of care aimed at identifying and measuring critical issues through the definition of criteria, indicators and standards to compare; feedback, returning summary reports of the results of performance evaluation to health professionals involved to promote change [
3,
4,
5,
6,
7].
A&F interventions produce improvements in professional practice of varying degrees [
3,
4,
5]. A Cochrane review published in 2012 concludes that A&F is effective with an absolute improvement of 4.3 percent (range interquartile 0.5; 16%) in adherence to evidence-based clinical practice recommendations. The change seems modest in absolute terms, but the cumulative gain resulting from repeated cycles of A&F can lead to large transformations. The ways in which A&F is implemented are widely varied among different studies and contexts, [
8,
9,
10] and, moreover, the scientific progress on these important aspects over the last 20 years has been minimal [
11,
12,
13].
The effectiveness of A&F can be increased if the feedback is posed by a colleague or supervisor, if it is performed more than once, if it is offered in both verbal and written form, and if it includes specific goals to achieve and an action plan to implement the changes. These and other recommendations on how to perform A&F optimally were the subject of a recent paper published by Brehaut et al. [
14].
The experience of the ASPIRE (
Action to Support Practice Implement Research Evidence) project in UK provided concrete evidence of effectiveness on a high burden disease and applied to larger populations through recommendation packages also based on the A&F tool, as it resulted in the management of chronic pain in primary care [
15].
The evidence on how well these recommendations is actually applied in A&F practice is still scarce [
16].
In Italy, A&F strategies are still underutilized in some contexts and rarely reported in scientific studies. In fact, of the 140 studies included in the 2012 Cochrane review [
11], only one was conducted in Italy. This poses clear limitations in terms of transferability of the efficacy results of the meta-analysis to the Italian context and supports the need to conduct experimental studies with a focus on research on general and context-specific hindering and facilitating factors.
As part of the EASY-NET project (NET-2016-02364191) [
17,
18], Work Package 1 (WP1) Lazio Emergency, through the Department of Epidemiology of the Regional Health Service (RHS) (namely DEP Lazio), compared effectiveness in improving the appropriateness and timeliness of emergency health interventions for acute myocardial infarction (AMI) and ischemic stroke between a structured A&F strategy and the voluntary consultation of many process and outcome indicators, annually updated, (“standard strategy”) through a dedicated regional web platform called P.Re.Val.E (“Programma Regionale Valutazione Esiti”—Regional Program for Outcomes and Processes Evaluation) [
2]. Therefore, in this “standard strategy”, the “feed-back” is returned to providers through a web publication, and no other initiatives are offered by DEP Lazio. A structured A&F intervention has been defined within the WP1 Lazio Emergency project, taking into account the most recent evidence in the field suggesting ways of optimizing these strategies [
14,
19,
20].
In 2021, 7766 hospitalizations for Acute Myocardial Infarction (AMI) and 3249 with ST elevation (STEMI) were reported in Lazio [
2]. Hospitalizations for IMA and STEMI appear to be progressively declining over the past decade and this is in line with national and international evidence [
1]. Also 30-day mortality, an indicator that reflects at least in part the quality of patient care provided, appears to be declining in recent years for both IMA and STEMI [
21]. In Lazio region, updated analyses to 2021 report a reduction from 9.7 percent to 7.6 percent and 11.1 percent to 8.8 percent, respectively, compared with 2012 [
2,
22].
The objective of this work is to perform a quantitative evaluation of effectiveness of an experimental A&F intervention as compared to the "standard strategy" in improving the adequacy and timeliness of emergency health interventions in patients with AMI and Stroke in the Lazio Region.
2. Materials and Methods
2.1. Study Design, Participants and Patients
The quantitative evaluation of effectiveness was conducted according to a prospective quasi-experimental pre-post intervention study with control group. The pre-and post-intervention periods considered are respectively the years 2021 and 2022.
The participants to the intervention, generally called “recipients”, are teams of professionals (clinical specialists and healthcare managers) involved in the emergency care for patients with AMI or stroke at hospitals in the Lazio region. Those exposed are hospitals that voluntarily participated in the intervention following a formal invitation. The control group is involved in usual reporting activities.
The effectiveness analysis was performed at patient level. We can therefore identify in the two periods, before and after the intervention, as exposed the patients admitted for AMI and/or stroke to the hospitals participating in the intervention and as controls the patients admitted for AMI and stroke to hospitals not participating in the intervention. (
Figure 1).
The pre- and post-intervention periods were compared, between exposed and controls using two process and/or outcome indicators of greatest interest per condition, listed below:
- -
30-days mortality after hospital admission in patients with AMI;
- -
proportion of PTCA (Percutaneous Transluminal Coronary Angioplasty) performed in STEMI (ST-Elevation Myocardial Infarction) patients within 90 minutes from the access to the hospital Emergency Room (ER);
- -
in-hospital mortality in patients with ischemic stroke;
- -
proportion of hospital readmissions within 30-days from the discharge in patients with ischemic stroke;
Details regarding these indicators (calculation formula, dimension, rational, period of calculation, and link to the calculation protocol) were reported in relevant sheet reported in supplementary material (
Table S1 and S2).
2.2. A&F Intervention and Control Group
Hospitals participating in the intervention were involved in the following periodic activities (6 months):
-the organization of periodic meetings to update project activities, present and discuss feedback contents;
-following each meeting, the feedback report was sent in different way (an overall main document and a hospital specific power point presentation) via e-mail to the hospital’s contact person (for IMA and/or stroke, respectively), together with a form for collecting information on audit meetings conducted after the feedback;
-a formal invitation to plan and conduct audit meetings following each feedback;
-the return of the form with information on the characteristics of the conducted Audits (date, participants, discussion of indicators, identification of improvement activities, Audit minutes etc) to the research group.
More detail regarding the A&F intervention implementation were reported in
Angelici et al. [
17].
The controls had web access to the results of the Regional Program for the Evaluation of Outcomes of Health Interventions (P.Re.Val.E.) [
2] curated by DEP, which annually publishes process and outcome indicators related to several chronic and acute conditions, including AMI and stroke. Using a specific function available through the platform, a healthcare company has the possibility of activating an Audit procedure also involving the Lazio DEP. Consultation is at the initiative of professionals, and the minimum level of aggregation available is by facility. Other available comparison data include data from other facilities, previous periods, and regional-level measures.
2.3. Data Sources
Pseudo-anonymized data collected through the health information systems (HIS) of the Lazio Region were used to calculated the indicators and to retrieve variables used as adjustment covariates in the analysis.
Specifically, the data sources included the Italian Hospital Discharge Registry (HDR), the Healthcare Emergency Information System (HEIS) and the Tax Registry. The HDR information system contains sociodemographic and clinical data systematically collected during each hospital ad-mission and discharge from facilities of the Lazio region, including the main and additional diagnoses and all the procedures carried out. Eligibility and exclusion criteria for the selection of the cohort of interest were defined according to the International Classification of Disease, Ninth Revision, Clinical Modification, (ICD-9-CM) codes [2019]. Codes for each indicator are reported in
Supplementary Materials Table S3.
An anonymous identification code, assigned from the HIS, was used as the key for the record-linkage procedure applied using the deterministic methodology. Data from the HDR were linked to data collected through the Information System of Health Emergency (HEIS) that routinely collects sociodemographic and clinical information about treatments and access to all the Emergency Department of hospitals in the Lazio region, the “Tax Registry” that includes information on deaths and the 2011 Census (Lazio Region Longitudinal Study) including information on patient’s education qualification [
23].
By integrating data from the different data sources described above, a socio-demographic and health-related profile is defined, allowing for patients’ clinical history for the five years preceding the admission of interest to be traced.
2.4. Variables in Analysis
At patient level demographic, socioeconomic and clinical variables were analyzed. Demographic data included sex and age categories. The level of education was used as a proxy of socioeconomic position and defined on the basis of the 2011 Census or, if not available from this source, of the information recorded in the HDR Education was analyzed according to the following categories: bachelor's degree, lower middle high school, middle high school, none or elementary, not stated.
The integration of different information sources allows us to trace the clinical history of patients in the five years preceding the hospitalization of interest (index hospitalization).
Clinical information (comorbidities, drugs) was searched in the index hospitalization and/or index admission to the hospital ER and/or in all hospitalizations or admissions to the ER in the previous five years (further details in
Table S1 and S2 of Supplementary material).
Finally, the type of hospital according to the Lazio Region adult emergency network [
24] was analyzed as context variables at the hospital level. The variable included the following categories: Emergency Admission Department level I (EADI), Emergency Admission Department level II (EADII) and hospitals with Emergency Room (ER) for AMI [
24]; Neurovascular Treatment Unit level I (NTUI), Neurovascular Treatment Unit level II (NTUII), hospital with a Neurovascular Treatment Team (NVT) and hospital without Neurovascular Treatment Team (noNVT) for stroke [
25].
2.5. Data Management and Statistical Analysis
Analyses were performed at patient level. For each analyzed indicator a dichotomous outcome variable was defined (Yes/No) (e.g. death or not death within 30-days from the hospital admission for AMI).
Patients admitted to hospital with volume of activities lower than 50 were excluded from the analyses to account for the relationship between volumes and outcomes [
26] and to obtain more reliable results.
Descriptive analyses of the demographic, socioeconomic, and clinical characteristics of patients according to the exposed/control status and according to the outcome, in pre and post intervention, were performed. P-values of association will be calculated by chi-square tests.
The effectiveness of the intervention was evaluated using Difference In Difference (D-I-D) models [
16,
17,
18,
19,
20] to compare the changes in the outcome occurring from pre to post intervention periods between the exposed and controls, taking into account changes in secular trends and controlling measured and unmeasured confounding factors. D-I-D models were implemented through generalized linear models with Binomial probability distribution and with identity as a link function. The estimates of D-I-D models were therefore presented in terms of the difference between absolute risks and measured in percentage points (pp).
Hospital and patient level characteristics are expected to confound or to modify the relationship between intervention and outcomes, and were evaluated as potential confounding factors or effect modifiers.
In particular, the univariate association of each single demographic, socioeconomic, clinical and contextual variable with the outcome of interest was tested, and a step-wise procedure was used to identify the set of covariates entering the final multivariate model. All data were analyzed using SAS version 9.4 (SAS Institute, Cary, NC, USA).
3. Results
Eighteen of 70 (25.7%) hospitals in the Lazio region participated to the intervention for a total of 29 clinical pathways: 16 for AMI and 13 for stroke. The participating hospitals are shown in supplementary material
Table S4.
3.1. Participating Hospitals
Thirty-one of 70 (44.3 %) hospitals with at least one AMI patient admission presented activity volumes of 50 admissions or more in the years 2021 and 2022. Fifteen of these (48.4%) were exposed to the A&F intervention, and 16 (51.6%) were not exposed (Descriptive details separately for exposed and controls is shown in supplementary material
Table S5). A total of 12196 AMI patients were admitted to hospitals included in the analysis, 5986 (49.1%) in 2021 and 6210 (50.9%) in 2022. Of these, 7002 (57.4%) were admitted to exposed and 5194 (42.6%) to the not exposed hospitals (
Table S5). Out of 59 hospitals that admitted at least one STEMI patient in the years 2021 and 2022, 20 presented activity volumes of at least 50, 12 (60%) exposed and 8 (40%) not exposed. A total of 5084 STEMI patients were included in the analysis, 2433 (47.8%) in 2021 and 2651 (52.5%) in 2022. Of these, 3272 (64%) were admitted in the exposed hospitals and 1812 (36%) in the controls (
Table S6).
Eighteen of 70 (25.7 %) hospitals with at least one ischemic stroke patient admission presented activity volumes of 50 admissions or more in the years 2021 and 2022. Ten of these (66.6%) were exposed to the A&F intervention, and 8 (44.4%) were not exposed (Descriptive details separately for exposed and controls is shown in supplementary material
Table S7). A total of 12196 AMI patients were admitted to hospitals included in the analysis, 5986 (49.1%) in 2021 and 6210 (50.9%) in 2022. Of these, 7002 (57.4%) were admitted to exposed and 5194 (42.6%) to the not exposed hospitals (
Table S5). A total of 5949 patients with ischemic stroke (who met the inclusion criteria for calculating the "In-hospital mortality for ischemic stroke") were admitted to hospitals included in the analysis, 2954 (49.7%) in 2021 and 2995 (50.3%) in 2022. Of these, 3793 (63.8%) were admitted to exposed hospitals and 2156 (36.2%) to the controls (
Table S7a). A total of 5453 ischemic stroke patients (who met the inclusion criteria for calculating the "30-day readmissions following an ischemic stroke") were analysed, 2685 (49.2%) in 2021 and 2768 (50.8%) in 2022. Of these, 3471 (63.7%) were admitted to the exposed hospitals and 1982 (36.3%) to controls (
Table S7b).
3.2. Patients’ Populations
3.2.1. AMI Patient’s Cohort
Of the total of 12196 AMI patients admitted to the hospitals included in the analyses in the two years (2021 and 2022), 70% were male with an average age of 69 years. Descriptive details are reported in supplementary material
Table S8.
No significant differences in terms of demographic and socioeconomic characteristics were found between exposed and controls patients, either in total and separately in the compared years 2021 and 2022. On the other hand, statistically significant differences were found between the frequency of some clinical conditions between exposed and controls including arterial hypertension (p=0.007), chronic kidney disease (p=0.006), previous coronary angioplasty (p=0.003), previous coronary artery bypass grafting (p=0.026). The hospital type of admission also showed significant differences (p<0.001) (supplementary material
Table S8,
Table S9,
Table S10).
In total (2021 and 2022), 839 (6.9%) patients admitted for AMI died within 30-days from the first hospital contact, 51.3% of these in the post-intervention year 2022 (
Table 1 and supplementary material
Table S11 and S12). Descriptive results are showed in
Table 1. Higher 30-days mortality was significantly associated with lower education, with being female and with being in the higher age class. Several clinical conditions were also associated with the outcome (
Table 1).
3.2.2. STEMI Patient’s Cohort
Of the total of 5984 STEMI patients admitted to the hospitals included in the analyses in the two years, 74% were male with an average age of 66 years. Descriptive details are reported in supplementary material
Table S13.
No significant differences in terms of demographic and socioeconomic characteristics were found between exposed and controls patients, either in total and separately in the compared years 2021 and 2022. Statistically significant associations emerge instead between the frequency of some clinical conditions between exposed and controls including obesity at index admission (p=0.009), Previous myocardial infarction (p=0.017), cardiomyopathies at index admission (p=0.001), and prior coronary angioplasty (p<0.001). The hospital type of admission also showed significant differences (p<0.001) (supplementary material
Table S13, S14, S15).
In total (2021 and 2022), 3077 (60.5%) patients underwent PTCA within 90 minutes of admission to the ED, 52% of these in the post-intervention year 2022. Descriptive results are showed in
Table 2. Higher proportion of PTCA performed in STEMI patients within 90 min of admission to the ED was significantly associated with lower education, with being male and being in the age class 58-65. Several clinical conditions were also associated with the outcome (
Table 2 and supplementary material
Table S16 and S17).
3.2.3. Ischemic Stroke Patient’s Cohort
The eligibility criteria for including patients in the stroke cohorts for calculating the two indicators considered are different, so the two cohorts are described separately.
30-days in hospital mortality after first hospital admission in patients with ischemic stroke
Of the total 5949 ischemic stroke patients admitted to the hospitals included in the analyses in the two years, 54% were male with an average age of 74 years. Descriptive details are reported in supplementary material
Table S18.
No significant differences in terms of demographic and socioeconomic characteristics were found between exposed and controls patients, either in total and separately in the compared years 2021 and 2022 except for educational qualification (p<0.001). Statistically significant differences were found between the frequency of some clinical conditions between exposed and controls, among them: obesity and anaemias at index admission (p=0.038 and p<0.001), coagulation defects at index admission (p=0.001), previous other forms of ischemic heart disease (p=0.002), previous not well-defined forms and complications of heart disease (p=0.004), rheumatic heart disease (p=0.001) and other previous cardiac conditions (p=0.003). Finally, also the type of emergency stroke network hospital of admission was associated with exposure status (p<0.001) (supplementary material
Table S18, S19, S20).
In total (2021 and 2022), 432 (7.3%) patients died in hospital within 30-days of first admission to the hospital, and of these, 46% in 2022. Descriptive results are showed in
Table 3. Higher 30 days mortality after first hospital admission was significantly associated with lower education, with being female sex and with being in the higher age class. Several clinical conditions were also associated with the outcome (
Table 3 and supplementary material
Table S21 and S22).
% of in-hospital readmissions within 30-days of discharge for ischemic stroke
Of the total of 12196 AMI patients admitted to the hospitals included in the analyses in the two years (2021 and 2022), 70% were male with an average age of 69 years. Descriptive details are reported in supplementary material
Table S23.
No significant differences in terms of demographic and socioeconomic characteristics were found between exposed and controls patients, either in total and separately in the compared years 2021 and 2022 except for educational qualification (p<0.001). Statistically significant differences were found between the frequency of some clinical conditions between exposed and controls including: anaemia and coagulation defects at index admission (p<0.001), prior myocardial infarction (p=0.034), other forms of ischemic heart disease (p=0.006), heart failure (p=0.019), rheumatic heart disease at index admission (p=0.001) and other cardiac conditions (p=0.019). Finally, also the type of emergency stroke network hospital of admission was associated with exposure status (p<0.001). (Supplementary material
Table S23, S24, S25).
In total (2021 and 2022), 392 (7.2%) patients experienced hospital readmission within 30-days of discharge for ischemic stroke, 54% of these in the post-intervention year 2022. Descriptive results are showed in
Table 4. Higher proportion of in hospital readmissions within 30-days of discharge for ischemic stroke was significantly associated with being in the higher age class (p= 0.005). Several clinical conditions were also associated with the outcome (
Table 4 and
Table S26 and S27).
3.3. Intervention Effectiveness Evaluation
Unadjusted and adjusted results of D-I-D models applied to compare each indicator between exposed and controls patients in the pre and post intervention periods are reported in
Table 5,
Table 6 and
Figure 2.
Regarding the AMI/STEMI pathway, the adjusted analyses of the 30-days mortality after the first hospital contact of patients with AMI showed a reduction by 0.22 pp from 2021 to 2022 in exposed and by 0.14 pp in controls with a DID estimate of -0.08 pp (CI95% -2.80; 2.65, p-value=0.956) showing a non-significant difference in favour of those exposed.
The proportion of patients with STEMI treated with PTCA within 90 minutes from the access to the ED increased by 1.31 pp from 2021 to 2022 in exposed and by 8.63 pp in controls with a DID estimate of -7.29 pp (CI95% -12.75; -1.83, p-value=0.009) showing a significant difference in favour of the controls.
This result contrasts with the expectation that the A&F intervention would lead to a significantly higher improvement in the exposed, so some sensitivity analyses were conducted with exclusion from the calculation of facilities (only one) that although they received periodic reporting (feedback) never returned information on the conducted audits, either through the proposed grid or in verbal form or at meetings, not adhering, therefore, to the protocol of the A&F intervention. The results of the sensitivity analysis confirm a tendency of the controls to show greater improvement than the exposed although the estimate loses significance: -4.38 (-10.0; 1.23), p-value= 0.1259.
For ischemic stroke patients, the adjusted DID model showed that in-hospital mortality was reduced by 0.74 pp from 2021 to 2022 in exposed and was increased by 0.26 pp in controls with a non-significant difference in favour of the exposed of -0.99 pp (CI95% -2.93; 0.95, p-value=0.315). Finally, 30-days readmissions after a hospitalization for ischemic stroke were reduced from 2021 to 2022 in the exposed by 0.54 pp and was increased by 3.25 pp in controls with a DID estimate of -3.80 pp (CI95% -6.57; -1.03, p-value=0.007) showing a significant difference in favour of those exposed (
Table 6,
Figure 2).
4. Discussion
This work evaluated effectiveness of an experimental A&F intervention compared to that of the "usual reporting strategy" (P.Re.Val.E) in improving the appropriateness and timeliness of emergency health interventions in patients with AMI and ischemic stroke, using process and outcome indicators. Although the intervention was focused on the in-hospital emergency pathways, all the organizations involved in the regional time-dependent network of Lazio [
27] were invited to collaborate.
The results are heterogeneous. Regarding the AMI/STEMI conditions, the percentage of PTCA performed in STEMI patients within 90 minutes improved both in exposed and unexposed and, although the improvement was significantly higher in the latter, this result became not significant at sensitivity analysis performed excluding hospitals that practically did not carry-out the intervention. No differences were found in the 30-day mortality after an AMI admission. Instead, regarding the ischemic stroke, the percentage of patients readmitted within 30 days from the discharge showed a significant reduction in exposed as compared to an increase in unexposed. Eventually, there was no significant effect on in-hospital mortality in patients with ischemic stroke. In general, and in line with the literature, changes in mortality are likely to need a longer follow-up to demonstrate the effectiveness of quality improvement interventions [
3].
This heterogeneity could be explained with different considerations. First of all, the post intervention period analyzed was short. The intervention started on February 2022 and lasted until September 2023, then in present analyses only the first 11 months of the post intervention period (2022) were included. Furthermore, the participating hospitals completed the first audit meeting in the first six months, then improving actions were implemented in the following period and, consequently, changes could begin to be seen during the last five months of the year. A further interpretation of the heterogeneity found could be that in 2021, differently from 2022, some hospitals, among both those exposed and unexposed, were still allocating entire wards to the care of COVID-19 patients, and this influenced strongly and in a variable way the internal processes and patients’ outcomes. This element could had led to an improvement of some indicators in those hospitals between 2021 and 2022, regardless of the intervention, because they were less affected by infection control measures. As the information about which hospitals and which wards included in the analyses were in this situation was not available, it was not possible to correct the association for these data.
A further explanation could be that also some hospitals in the non-exposed group could have carried-out other quality improvement activities independently of the EASY-NET project and, therefore, this could have influenced the analysis. Since this information was not available, it was not possible to adjust analyses also for this factor.
Finally, it is important to consider that the intervention was focused only on the in-hospital part of the pathway without taking into account the how the entire network works. In fact, within the network, the emergency service (ARES118- Regional Health Emergency Company) transports patients to the most appropriate hospital emergency room based on their clinical severity and this could influence the comparison between exposed (including a large percentage of hospitals delivering emergency care of higher intensity) and controls (e.g. Supplementary material
Table S8). Nevertheless, to account for this limitation, the analyses were adjusted by hospital type.
Despite quantitative results seem not conclusive and further analyses considering a longer follow up are needed, additional valuable results deserve to be discussed regarding the positive impact that the intervention had towards the creation of links and opportunities for discussion and benchmarking between professionals from different disciplines (cardiology, neurology, emergency, health management, epidemiology), hospitals and settings. They contributed to improve the quality of the data collected in the participating institutions that feed the health information flows. In fact, during the periodic meetings the professionals involved, researchers and regional referents could discuss results of indicators that appeared not matching their expectations. For example, in different cases this was due to the incorrect use of codes or mistakes in marking the time of procedures during the data entry process. Specific audits on data quality were then organized [
28]. This fact could also impact on the effectiveness evaluation because the follow-up did not yet include a period following the implementation of data quality improving actions.
Another plus arising from the intervention was the contribution that all the participants made to the production of audit support materials (reports and audit form) agreed among professionals and regional referents, that could be used in the daily practice beyond the end of the research project. As recent publications suggested [
14,
19], involving recipients since the A&F design (indicators selection and definition, feedback materials and timing), integrating both verbal and written feedback (reports document and in person meeting), giving them the opportunities to periodically discuss in social occasions their results, problems and solutions and stimulating a peer collaboration could enhance motivation and prevent discontinuation. All the hospitals but one participated in all the scheduled activities and actively contributed to the meeting discussions. Furthermore, hospitals could organize periodic audit meetings independently by the research group in order to adapt as much as possible activities to the specific context of hospitals differing in size, complexity, volume of activities and process organization.
In an era of increasing complexity and demand for value, the insights generated aim to enhance evidence-based practice and contribute to the ongoing evolution of healthcare delivery models. The provision of feedback and the establishment of cyclical audit processes could support the effective implementation of networks that improve the appropriateness and timeliness of emergency health interventions for patients with time-dependent conditions.
The proposed intervention involved all actors of the time-dependent emergency network in the context of the Lazio region, although the focus of the activities was the in-hospital pathway. There is evidence that clinical networks can improve the delivery of healthcare though there are few high-quality quantitative studies of their effectiveness [
29]. Organizations in such networks need to collaborate and coordinate their actions to achieve their common purpose. They also need to align goals, balance power, manage conflict, monitor performance, and hold members accountable for network-level outcome [
30].
Current performance measurement systems in Italy provide feedback on time-dependent conditions of care [
1,
2,
31,
32], but rarely allow the evaluation of interventions at the network level. Further opportunities for research in this filed should adopt a network vision in order to evaluate and to improve the entire emergency pathway that could largely influence critical nodes also in other parts of the patients’ journey.
5. Conclusions
Results show, for the first time in Italy, the impact of A&F interventions in an emergency environment using aggregated data from hospitals involved in the Lazio Region Emergency network. The provision of feedback and the establishment of cyclical audit processes could support the effective implementation of networks that improve the appropriateness and timeliness of emergency health interventions for patients with time-dependent conditions.
Supplementary Materials
The following supporting information can be downloaded at the website of this paper posted on Preprints.org. Table S1. Detailed cardiovascular area (AMI/STEMI) indicators sheets. Table S2. Detailed cerebrovascular area indicators sheet. Table S3. ICD-9-CM codes. Table S4. List of participating facilities. Figure S1. Flow chart of inclusion of Lazio facilities who have in charge AMI patients. Figure S2. Flow chart of inclusion of Lazio facilities who have in charge STEMI patients. Figure S3. Flow chart of inclusion of Lazio facilities who have in charge ischemic stroke patients. Table S5. Distribution of patients belonging to the IMA cohorts by year of analysis and facility. Table S6. Distribution of patients belonging to the STEMI cohorts by year of analysis and facility. Table S7. Distribution of patients belonging to stroke cohorts by year of analysis and facility (a and b). Table S8. Characteristics of patients included in the AMI cohort in 2021 and 2022 from participating facilities according to A&F intervention exposure status. Table S9. Characteristics of patients included in the AMI cohort in 2021 (PRE) from participating facilities according to A&F intervention exposure status. Table S10. Characteristics of patients included in the AMI cohort in 2022 (POST) from participating facilities according to A&F intervention exposure status. Table S11. Characteristics of patients included in the AMI cohort in 2021 (PRE) from participating facilities by 30-days mortality after first hospital admission in patients with AMI (Yes/No). Table S12. Characteristics of patients included in the AMI cohort in 2022 (POST) from participating facilities by 30-days mortality after first hospital admission in patients with AMI (Yes/No). Table S13. Characteristics of patients included in the STEMI cohort in 2021 and 2022 from participating facilities according to A&F intervention exposure status. Table S14. Characteristics of patients included in the STEMI cohort in 2021 (PRE) from participating facilities according to A&F intervention exposure status. Table S15. Characteristics of patients included in the STEMI cohort in 2022 (POST) from participating facilities according to A&F intervention exposure status. Table S16. Characteristics of patients included in the STEMI cohort in 2021 (PRE) from participating facilities according to performing of PTCA within 90 minutes of admission to the ER (Yes/No). Table S17. Characteristics of patients included in the STEMI cohort in 2022 (POST) from participating facilities according to performing of PTCA within 90 minutes of admission to the ER (Yes/No). Table S18. Characteristics of patients included in the ischemic stroke cohort (in hospital mortality) in 2021 and 2022 from participating facilities according to A&F intervention exposure status. Table S19. Characteristics of patients included in the ischemic stroke cohort (in hospital mortality) in 2021(PRE) from participating facilities according to A&F intervention exposure status. Table S20. Characteristics of patients included in the ischemic stroke cohort (in hospital mortality) in 2022 (POST) from participating facilities according to A&F intervention exposure status. Table S21. Characteristics of patients included in the ischemic stroke cohort in 2021 (PRE) from participating facilities by in hospital 30-days mortality since first hospital admission in patients with ischemic stroke (Yes/No). Table S22. Characteristics of patients included in the ischemic stroke cohort in 2022 (POST) from participating facilities by in hospital 30-days mortality since first hospital admission in patients with ischemic stroke (Yes/No). Table S23. Characteristics of patients included in the ischemic stroke cohort (hospital readmissions) in 2021 and 2022 from participating facilities according to A&F intervention exposure status. Table S24. Characteristics of patients included in the ischemic stroke cohort (hospital readmissions) in 2021(PRE) from participating facilities according to A&F intervention exposure status. Table S25. Characteristics of patients included in the ischemic stroke cohort (hospital readmissions) in 2022 (POST) from participating facilities according to A&F intervention exposure status.
Author Contributions
Conceptualization, L.A., A.A. and N.A.; methodology, L.A., A.A., L.P., P. C. and N.A.; data curation, L.A., A.A. and A.M.; formal analysis, L.A.; writing – original draft, L.A., A. A. and N. A.; writing – review & editing, L.A., A.A., C.A., L. P., P.C., A.D.B., E.L.G., A.M., S.F., M.D. and N.A.; supervision, A.A., M.D. and N.A.; project administration, L.A., A.A., C.A., M.D. and N.A.; funding acquisition, S.F., M.D. and N.A. All authors read and approved the final manuscript and accepted personal responsibility for their contributions and ensured the accuracy and integrity of all parts of the work, even those in which they were not personally involved.
Funding
This study was supported by the Italian Ministry of Health and Lazio Region (EASY‐NET project code: NET‐2016‐02364191). The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Institutional Review Board Statement
This study was carried out The Department of Epidemiology (DEP) is regulated by the Lazio Regional Health Service in managing and analyzing data from the administrative information systems for epidemiological research, according to current regional law (Lazio, Italy). The DEP has access to anonymized data taken from the Regional Health Infor mation Systems and it is not possible to trace patient identity. The DEP works in synergy with the Directorate for Health and Social Care Integration of the Lazio Regional Health Service. As a result, the DEP is entitled to use the data provided by the Health Information System Unit of the Lazio region for health and scientific purposes. This article reports on research developed within the P.Re.Val.E. [
2]. Furthermore, the DEP has been identified as being responsible for the development of the P.Re.Val.E. and for all aspects related to its technical and scientific implementation, according to current regional law. For these reasons, it is exempt from the requirement for approval by an Institutional Review Board in this setting.
Informed Consent Statement
Not applicable.
Data Availability Statement
Data related to the findings reported in our manuscript are available to all interested researchers upon reasonable request and with the permission of the Regional Department because of stringent legal restrictions regarding the privacy policy on personal information in Italy (national legislative decree on privacy policy n. 196/30 June 2003). For these reasons our dataset cannot be made available on public data deposition.
Acknowledgments
This work was produced as part of the activities of the EASY-NET research group "Audit & Feedback. Effectiveness of Audit & Feedback strategies to improve healthcare practice and equity in various clinical and organizational settings (EASY-NET)" (project code: NET-2016-02364191), funded by the Ministry of Health and co-funded by the participating regions (Lazio, Friuli Venezia Giulia, Piedmont, Emilia-Romagna, Lombardy, Calabria). We thank all researchers involved in the EASY-NET network program (
http://easy-net.info/progetto-easy-net-migliorare-la-qualita-di-assistenza-con-audit-feedback/) and in particular all
Members of the EASY-NET: WP1 Lazio research group: Anna Acampora, Nera Agabiti (PI); Michela Alagna, Laura Angelici, Carmen Angioletti, Maria Balducci, Giulia Cesaroni, Paola Colais, Marina Davoli, Melissa D’Agostino, Antonio Giulio de Belvis, Mirko Di Martino, Danilo Fusco, Domenico Antonio Ientile, Adele Lallo, Francesca Mataloni, Ursula Kirchmayer, Amina Pasquarella, Luigi Pinnarelli, Chiara Sorge, Salvatore Soldati, Sara Farchi and Sergio Ribaldi. Finally, we thank also all participating facilities.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Programma Nazionale Esiti – Edition 2021. Available at: https://www.agenas.gov.it/comunicazione/primo-piano/2005-pne-2021-agenas-presenta-i-risultati-al-ministero-della-salute. (accessed on 10 September 2022). (accessed on 10 September 2022).
- P.Re.Val.E. (Programma Regionale Valutazione Esiti”—Regional Program for Outcomes and Processes Evaluation) edition 2022 and 2023 web sites. Available at: https://www.dep.lazio.it/prevale2022/ ; https://www.dep.lazio.it/prevale2023/. (accessed on 15 September 2022). (accessed on 15 September 2022).
- Ivers, N.; Jamtvedt, G.; Flottorp, S.; Young, J.M.; Odgaard-Jensen, J.; French, S.D.; O’Brien, M.A.; Johansen, M.; Grimshaw, J.; Oxman, A.D. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Syst. Rev. 2012, 6, 1465–1858. [Google Scholar] [CrossRef] [PubMed]
- Desveaux, L.; Ivers, N.M.; Devotta, K.; Ramji, N.; Weyman, K.; Kiran T. Unpacking the intention to action gap: a qualitative study understanding how physicians engage with audit and feedback. Implement. Sci. 2021, 16(1):19. [CrossRef]
- Moore, L.; Guertin, J.R.; Tardif, P.A.; Ivers, N.M.; Hoch, J.; Conombo, B.; Antony, J.; Stelfox, H.T.; Berthelot, S.; Archambault, P.; Turgeon, A.; Gandhi, R.; Grimshaw, J.M. Economic evaluations of audit and feedback interventions: a systematic review. BMJ Qual. Saf. 2022, 31(10), 754–767. [Google Scholar] [CrossRef] [PubMed]
- Schondelmeyer, A.C.; Bettencourt, A.P.; Xiao, R.; Beidas, R.S.; Wolk, C.B.; Landrigan, C.P.; Brady, P.W.; Brent, C,R,; Parthasarathy, P.; Kern-Goldberger, A.S.; Sergay, N.; Lee, V.; Russell, C.J.; Prasto, J.; Zaman, S.; McQuistion, K.; Lucey, K.; Solomon, C.; Garcia, M.; Bonafide, C.P. Pediatric Research in Inpatient Settings (PRIS) Network. Evaluation of an Educational Outreach and Audit and Feedback Program to Reduce Continuous Pulse Oximetry Use in Hospitalized Infants With Stable Bronchiolitis: A Nonrandomized Clinical Trial. JAMA Netw Open. 2021 Sep 1, 4(9): e2122826. [CrossRef]
- Goulao, B.; Scott, C.; Black, I.; Clarkson, J.; McArthur, L.; Ramsay, C.; Young, L.; Duncan, E. Audit and feedback with or without training in-practice targeting antibiotic prescribing (TiPTAP): a study protocol of a cluster randomised trial in dental primary care. Implement Sci. 2021 Mar 30;16(1):32. [CrossRef]
- Lau, R.; Stevenson, F.; Ong, B.N.; Dziedzic, K.; Treweek, S.; Eldridge, S.; Everitt, H.; Kennedy, A.; Qureshi, N.; Rogers, A.; Peacock, R.; Murray, E. Achieving change in primary care-effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ Open. 2015 Dec 23, 5(12):e009993. [CrossRef]
- Tuti, T.; Nzinga, J.; Njoroge, M.; Brown, B.; Peek, N.; English, M.; Paton, C.; van der Veer, S.N. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implement Sci. 2017 May 12,12(1):61. 12 May. [CrossRef]
- Chatzopoulou, M.; Kyriakaki A.; Reynolds L. Review of antimicrobial resistance control strategies: low impact of prospective audit with feedback on bacterial antibiotic resistance within hospital settings. Infect. Dis. (Lond) 2021, 53(3), 159- 68. [CrossRef]
- Ivers, N.M.; Grimshaw, J.M.; Jamtvedt, G.; Flottorp, S.; O’Brien, M.A.; French, S.D.; Young, J.; Odgaard-Jensen, J. Growing literature, stagnant Science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J. Gen. Intern. Med. 2014, 29, 1534–1541. [Google Scholar] [CrossRef]
- Grimshaw, J.M.; Ivers, N.; Linklater, S.; Foy, R.; Francis, J.J.; Gude, W.T.; Hysong, S.J. Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Qual. Saf. 2019, 28(5), 416–23. [Google Scholar] [CrossRef] [PubMed]
- Foy R.; Skrypak M.; Alderson S.; Ivers N.M.; McInerney B.; Stoddart J.; Ingham J.; Keenan D. Revitalising audit and feedback to improve patient care. BMJ 2020, 368:m213. [CrossRef]
- Brehaut, J.C.; Colquhoun, H.L.; Eva, K.W.; Carroll, K.; Sales, A.; Michie, S.; Ivers, N.; Grimshaw, J.M. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann. Intern. Med. 2016, 164, 435–441. [Google Scholar] [CrossRef]
- Willis TA, Hartley S, Glidewell L, Farrin AJ, Lawton R, McEachan RRC, et al. Action to Support Practices Implement Research Evidence (ASPIRE): protocol for a cluster-randomised evaluation of adaptable implementation packages targeting ‘high impact’ clinical practice recommendations in general practice. Implement Sci. 29 febbraio 2016;11:25.
- Foster, M.; Presseau, J.; Podolsky, E.; McIntyre, L.; Papoulias, M.; Brehaut, J.C. How well do critical care audit and feedback interventions adhere to best practice? Development and application of the REFLECT-52 evaluation tool. Implement. Sci. 2021, 16(1), 81. [Google Scholar] [CrossRef] [PubMed]
- Angelici, L.; Angioletti, C.; Pinnarelli, L.; Colais, P.; de Mattia, E.; Agabiti, N.; Davoli, M.; Acampora A. EASY-NET Program: Methods and Preliminary Results of an Audit and Feedback Intervention in the Emergency Care for Acute Myocardial Infarction in the Lazio Region, Italy. Healthcare (Basel). 2023 Jun 5;11(11):1651. [CrossRef]
- EASY-NET Project Web Site. Available online: https://easy-net.info/ (accessed on 15 July 2022).
- Colquhoun, H.L.; Carroll, K.; Eva, K.W.; Grimshaw, G.M.; Ivers, N.; Michie, S.; Sales, A.; Brehaut, J.C. Advancing the literature on designing audit and feedback interventions: Identifying theory-informed hypotheses. Implement. Sci. 2017, 12, 117. [Google Scholar] [CrossRef] [PubMed]
- Bourrée, F.; Michel, P.; Salmi, L.R. Méthodes de consensus: Revue des méthodes originales et de leurs grandes variantes utilisées en santé publique [Consensus methods: Review of original methods and their main alternatives used in public health]. Rev. Epidemiol. Sante Publique 2008, 56, 415–423. [Google Scholar] [CrossRef] [PubMed]
- Laforgia, P.L.; Auguadro, C.; Bronzato, S.; Durante, A. The reduction of mortality in acute myocardial infarction: From bed rest to future directions. Int. J. Prev. Med. 2022, 13, 56. [Google Scholar].
- P.Re.Val.E Edition 2022 Web Site. Volume of Hospitalization for AMI from 2012 to 2021. Available online: https://www.dep.lazio.it/prevale2022/risultati/tipo5/home_tipo5.php?ind=122&tipo=5&area=1 (accessed on 15 September 2022).
- Angelici, L.; Sorge, C.; Di Martino, M.; Cappai, G.; Stafoggia, M.; Agabiti, N.; Girardi, E.; Lanini, S.; Nicastri, E.; Davoli, M.; Cesaroni. G. Incidence of SARS-CoV-2 Infection and Related Mortality by Education Level during Three Phases of the 2020 Pandemic: A Population-Based Cohort Study in Rome. J Clin Med. 2022, 7;11(3):877. /: https. [CrossRef]
- Rete emergenza ospedaliera – Afferenze rete emergenza ospedaliera adulti. https://www.regione.lazio.it/enti/salute/sistemi-emergenza/rete-emergenza-ospedaliera (accessed on 15 September 2023).
- Rete ICTUS. https://www.regione.lazio.it/enti/salute/sistemi-emergenza/rete-ictus- (accessed on 15 September 2023).
- Amato, L.; Fusco, D.; Acampora, A.; Bontempi, K.; Rosa, A.C.; Colais, P.; Cruciani, F.; D'Ovidio, M.; Mataloni, F., Minozzi, S.; Mitrova, Z.; Pinnarelli, L.; Saulle, R., Soldati, S.; Sorge, C.; Vecchi, S.; Ventura, M.; Davoli, M. Volume and health outcomes: evidence from systematic reviews and from evaluation of Italian hospital data. Epidemiol Prev. 2017, 41(5-6 (Suppl 2)):1-128. [CrossRef]
- Programmazione della rete ospedaliera 2021-2023 in conformità agli standard previsti nel DM 70/2015 https://www.regione.lazio.it/sites/default/files/documentazione/SAL_DD_G01328_10_02_2022_Allegato_1.pdf (accessed Dec. 28, 2023).
- P.Re.Val.E. (Programma Regionale Valutazione Esiti” - Regional Program for Outcomes and Processes Evaluation- Strumenti per Audit) edition 2022 and 2023 web sites. Available at: https://www.dep.lazio.it/prevale2022/audit/audit_qualita1.php?ind=1;https://www.dep.lazio.it/prevale2023/audit/audit_qualita1.php?ind=1.(accessed on 16 September 2023).
- The BC Emergency Medicine Network: Evaluation approach and early findings. British Columbia Medical Journal. https://bcmj.org/articles/bc-emergency-medicine-network-evaluation-approach-and-early-findings (accessed Dec. 27, 2023).
- Evans, J.M.; Commisso, E.; Grudniewicz A.; Im, J.; Veillard, J.; Richards, G. Managing the performance of healthcare networks: a ‘dance’ between control and collaboration. Public Manag. Rev. 2023. /: https. [CrossRef]
- Vola, F.; Benedetto, V.; Vainieri M.; Nuti S. The Italian interregional performance evaluation system. Res. Heal. Serv. Reg. 2022, 11, 1(1): 1–14. [CrossRef]
- Il sistema di monitoraggio. https://www.salute.gov.it/portale/lea/dettaglioContenutiLea.jsp?lingua=italiano&id=4744&area=Lea&menu=monitoraggioLea (accessed Dec. 28, 2023).
|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).