Failure to rescue (FTR) is defined by the number of deaths among patients experiencing major complications after surgery. In this report we analyze FTR and apply a cumulative sum control chart (CUSUM) methodology for monitoring performance in a large series of operated lung carcinoma patients.
MethodsProspectively stored records of cases undergoing anatomical lung resection in one center were reviewed. Postoperative adverse events were coded and included as a binary variable (major, or minor complications). The occurrence of 30-day mortality was also recorded. Patients dying after suffering major complications were considered as FTR. Risk-adjusted CUSUM graphs using EuroLung1 and 2 variables were constructed for major complications and FTR. Points of plateauing or trend inversion were checked to detect intentional or non-adverted changes in the process of care.
Results2237 cases included. 9.1% cases suffered major complications. The number of cases considered as failures to rescuing was 46 (2.1% of the total series and 22.5% of cases having major complications). The predictive performance of EuroLung1 and 2 models was as follows: EuroLung1 (major morbidity) C-index 0.70 (95%CI: 0.66–0.73); EuroLung2 (applied to FTR) C-index 0.81 (95%CI: 0.750.87). CUSUM graphs depicted improvement in rescuing complicated patients after case 330 but no improvement in the rate of non-complicated cases until case 720.
ConclusionsFTR offers a complementary view to classical outcomes for quality assessment in Thoracic Surgery. Our study also shows how the analysis of FTR on time series can be applied to evaluate changes in team performance along time.
El fallo en el rescate (FTR, por sus siglas en inglés) se define por el número de muertes entre los pacientes que experimentan complicaciones graves tras la cirugía. En este informe analizamos el FTR y realizamos gráficos de control de suma acumulada (CUSUM, por sus siglas en inglés) para monitorizar los resultados en una serie de gran tamaño de pacientes operados de carcinoma de pulmón.
MétodosSe revisaron los datos almacenados prospectivamente de los casos de resección pulmonar anatómica en un centro. Se codificaron los eventos adversos postoperatorios y se incluyeron como una variable binaria (complicaciones graves o menores). También se registró la mortalidad a los 30 días. Los pacientes que murieron después de sufrir complicaciones mayores se consideraron FTR. Se elaboraron gráficos CUSUM ajustados al riesgo, utilizando las variables EuroLung1 y 2, para el análisis de las complicaciones graves y el FTR. Se analizaron los puntos de estancamiento o inversión de la tendencia para detectar los posibles cambios voluntarios o inadvertidos en el plan de cuidados.
ResultadosSe incluyeron 2.237 casos. El 9,1% sufrieron complicaciones graves. El número de casos que se consideraron como fallos en el rescate fue de 46 (2,1% del total de la serie y 22,5% de los casos con complicaciones graves). El rendimiento predictivo de los modelos EuroLung1 y 2 fue el siguiente: EuroLung1 (morbilidad grave) índice C: 0,70 (IC 95%: 0,66-0,73); EuroLung2 (aplicado al FTR) índice C: 0,81 (IC 95%: 0,75-0,87). Los gráficos de CUSUM mostraron una mejora en el rescate de pacientes con complicaciones después del caso 330, pero ninguna mejora en la tasa de casos sin complicaciones hasta el caso 720.
ConclusionesEl FTR ofrece una visión complementaria a los resultados clásicos para la valoración de la calidad en la cirugía torácica. Nuestro estudio también muestra cómo se puede aplicar el análisis de FTR en series temporales para evaluar los cambios en el rendimiento del equipo a lo largo del tiempo.
Quality remains a serious concern within health care systems. Six areas or dimensions are usually considered to define health care quality: effectiveness (provided care is based in evidences and resulting in better outcomes for the served population), efficiency (health care is sustainable), accessibility, patient-centeredness (provided care considers patient preferences), equity (quality does not change because of population characteristics such as gender, race or social status), and safety (risks and harms are measured and minimized).1
Health providers, such hospitals and clinical services or units are concerned to ensure that they provide the highest possible standard of care meeting the needs of patients and are obliged to periodically auditing all dimensions of quality, and particularly those easily amenable to control: effectiveness and safety. In surgical services, safety is usually evaluated measuring and benchmarking the risk-adjusted rates of morbidity and mortality2 and, in the case of the European Society of Thoracic Surgeons (ESTS), these parameters are included in a set of structure and process-related variables to construct the Composite Performance Score (CPS) intended to measure the quality of care after lung resection.3
Besides morbidity and mortality, failure to rescuing complicated patients has been proposed by Silber et al.4 as a measure of hospital performance. FTR is defined as the number of deaths among patients experiencing an acute event, such as a postoperative complication. According to Silber et al.4 patient's characteristics drive the occurrence of complications, whereas hospital characteristics are associated with FTR.
Although FTR has been already endorsed as hospital-level quality metric in some institutions,5 it is not considered as an indicator of the surgical care quality by the ESTS Database6 and it is not included as one of the variables in the CPS yet. We hypothesize that adding that metric to the ESTS Institutional Quality Certification Program7 would lead to better analyzing quality variations among hospitals as it has been described in other settings.8
On the other hand, quality assessment based on cross-sectional analysis offers a good picture of the aggregate outcomes but does not reflect the team progress along time unless specific periods are evaluated and compared. On the contrary, cumulative sum (CUSUM) techniques offer the possibility of checking a process along time and knowing if its quality is kept constant, improves or deteriorates.9 In the last case, corrective measures can be implemented, and their efficacy investigated.
The aim of this study is describing the occurrence of postoperative adverse events and FTR in a single institution reporting to the ESTS Database and applying a methodology of sequential analysis of quality (using Euroloung1 and 2 for calculating the risk-adjusted CUSUM) to evaluate changes in performance along time.
MethodsProspectively stored records of consecutive cases undergoing scheduled anatomical lung resection (pneumonectomy, bilobectomy, lobectomy or anatomic segmentectomy) in one single center from January 1994 to December 2017 were reviewed. The occurrence of 30-day mortality was recorded in the database. In cases dying outside of the Hospital, date of death was obtained from referral doctors or from the next-of-kin. Complications were defined in advance and agreed by all team members uploading data in the database. Postoperative adverse events were retrospectively recoded according to the Dindo et al. systematic classification of postoperative morbidity10 and included as a binary variable (major, including IIIA to V, or minor complications, including I and II classes of the score). Two authors (GV and NN) re-classified adverse events to fit the Dindo et al.10 classification. FTR was defined as the occurrence of 30-day mortality in patients suffering major postoperative complications. In cases dying after hospital discharge, records were re-assessed to ascertain if patient's death was a late consequence of in-hospital complications.
Patient's individual risk of major complications was calculated according to the European risk model for morbidity (EuroLung1).11 For that purpose, the following variables were reviewed from the unit's database: patient's gender and age; predicted postoperative forced expiratory volume in 1s (ppoFEV1); previous episode of coronary artery disease (CAD); cerebrovascular disease (CVD); or chronic kidney disease; surgical approach (VATS or open); and extended resection. For the risk of failure to rescuing, the European model of hospital mortality (EuroLung2) was applied. EuroLung2 includes the following variables: patient's gender and age; ppoFEV1; CAD; CVD; body mass index; surgical approach; pneumonectomy; and extended resection. Predictive capacity of both models was checked measuring C-indexes (and their 95%CI) on non-parametric ROC curves. Data accuracy and completeness were assured by double-checking data. First quality control was performed by the surgeon in charge of discharging the patient and the second one by the team quality manager [MJ] at the time of sending patient's charts to the hospital registry. Cases with missing data were excluded from the analysis. For risk modeling and ROC curves we used Stata/IC 15.1 software. Records were arranged on time series in ascending order of their date of surgery and risk-adjusted CUSUM graphs were constructed for major complications and FTR using Excel 2016.
Potential changes in the process of patients’ care influencing plateauing or inversion of trending in CUSUM graphs were discussed and agreed by team members.
ResultsThe series consisted in 2341 cases. 104 records (4%) were excluded from the analysis due to missing data. The final set included 2237 cases (300 pneumonectomies, 134 bilobectomies, 1717 lobectomies and 86 segmentectomies). Any postoperative adverse event was recorded in 758 cases (33.8%), and 204 (9.1%) cases suffered major complications. Overall 30-day mortality was 52 cases, 9 patients died after hospital discharge and 41 were in-hospital mortality. Of the 9 patients dying after hospital discharge, 6 showed no postoperative complication while in-hospital. In these cases, the cardiac or cardiovascular event causing death happened without relationship to any previous event. Therefore, according to the definition, the number of cases considered as failures to rescuing was 46 (2.1% of the whole series of cases and 22.5% of cases with major complications). In Table 1 we are presenting the prevalence of different grades of complications in the series and the occurrence of 30-day mortality in each subgroup of patients. The predictive performance of EuroLung1 and 2 models was as follows: EuroLung1 (major morbidity) C-index 0.70 (95%CI: 0.66–0.73); EuroLung2 (applied to FTR) C-index 0.81 (95%CI: 0.75–0.87) (Fig. 1).
Prevalence of complications in the series according to Dindo et al. classification of adverse postoperative events10 and 30-day mortality in each group of cases.
| Grade of complication | N of cases (%) | 30-Day deathN (%) |
|---|---|---|
| 0 | 1479 (66.1) | 6 (0.4)a |
| I | 219 (9.8) | 0 |
| II | 335 (15) | 0 |
| IIIa | 89 (4) | 0 |
| IIIb | 49 (2.2) | 1 (2) |
| IVa | 20 (0.9) | 2 (10) |
| IVb | 3 (0.1) | 0 |
| V | 43 (1.9) | 43 (100) |
CUSUM graphs are represented in Fig. 2. Cases are arranged by date of surgery on the X-axis. Unsatisfactory performance occurs until dashed arrows that represent the point from which a better team performance is observed. Cases 330 for FTR and 720 for major complications were identified as the lowest points of the curves. From these points both curves showed a clear and steadily upward tendency especially for FTR. For major complications a second low performance area is showed that was quickly detected and reversed (cases 1750–1900).
DiscussionMain findings of the study
Three results can be highlighted. First, 2.1% of the whole series of cases and 22.5% of cases with major complications died after surgery because morbidity could not be controlled. Three out of 46 cases occurred after hospital discharge raising the concern that complicated patients need a special follow-up, once at home, in an attempt for early recognition of any possible deterioration. Second, EuroLung1 and 2 showed a good performance when applied to our series of cases. To note that EuroLung2 developed to evaluate mortality is well-adjusted to FTR. However, it would be important having a specific model for it instead. Third, CUSUM charts clearly showed the team performance along the years. A retrospective audit of patient management practices identified case number 330 as coincident with the implementation of institutional guidelines for standard perioperative care (dealing early detection and ward-initiated therapy of most frequent complications after lung resection). Around case number 720, a more aggressive protocol of perioperative physiotherapy was implemented. The second depression in the morbidity performance is related to the learning curve for VATS procedures but it was quickly detected, and discussion and implementation of specific measures changed the trending.
Relevance of controlling outcomes in health care servicesSocial and financial changes all around the World are obliging governments to restructure health care services increasing accountability, cost effectiveness, sustainability, and quality improvement strategies.12 Developing systems helping to monitor the performance of health care providers must be aimed at institutional, national and international levels. In this context, and to gain solid evidence in the field of hospital performance, the World Health Organization (WHO) Regional Office for Europe launched in 2003 the Performance Assessment Tool for quality improvement in Hospitals (PATH), designed to support hospitals implementing initiatives for quality control and translating them into actions for continuing improvement.12 PATH includes six interrelated dimensions, safety being one of them. Patient's safety is defined as “the dimension of performance, wherein a hospital has the appropriate structure, and uses care delivery processes that measurably prevent or reduce harm or risk to patients”.12 In that definition, the three classical measurable components of quality in Medicine (structures, processes and outcomes)13 are included. One of the measurable performance indicators for patient's safety proposed by PATH12 is hospital mortality for selected tracer conditions and procedures.
Current initiatives for quality control and improvement promoted by the ESTSHospital mortality and postoperative morbidity are the outcome indicators included in the CPS promoted by the ESTS Audit and Clinical Excellence Committee.3 This performance score serves as one of the fundamentals for the ESTS Institutional Quality Certification Program.14 Although hospital mortality underestimates 30-day mortality15 and, of course, all mortality occurring as a late consequence of a surgical procedure, it is a robust variable easily measurable and accessible from administrative databases. At the CPS, outcomes are measured in a cross-sectional approach, which is useful for establishing comparisons between units when standard values are known, especially if the process is evaluated on the same time for different health care providers.16 Quality assessment based on cross-sectional analysis offers a good picture of the aggregate outcomes but does not reflect the team progress along time unless specific periods are evaluated and compared. On the contrary, CUSUM techniques offer the possibility of checking a process along time and knowing if its quality is kept constant, improves or deteriorates.17 In the last case, corrective measures can be implemented, and their efficacy investigated.
Why measuring FTR in surgical settingsMeasuring FTR in health care comes from the concept of high reliability organizations (HRO) in the industry. The main characteristic of HRO is not its failure rate, but its effective management of failures. In surgery, not every complication can be prevented but it is expected from a surgical team to be able to identify and treat complications as soon as they occur. Weick and Sutcliffe18 identified resilience as a defining characteristic of HROs. Members of a resilient organization understand that the unexpected cannot always be avoided. Consequently, they are aware of complications and work hard to diagnosing them sooner and treating them whenever possible.19 Including failure to rescuing patients to the ESTS quality improvement program would offer the possibility of benchmarking hospital performance rather than complexity8 as it has been shown in one of the few papers studying the rates of FTR at different hospitals. In that investigation, the authors found that failure-to-rescue rates were significantly higher in high mortality centers compared to those with low mortality.20 A similar conclusion can be found in papers analyzing larger series of cases.21
The correlation of several variables with the rates of failure to rescuing patients have been studied in recent years; as an example, patient's advanced age (75 and over) represents a negative influence on the probabilities of rescuing after the first complication occurs in emergency surgery.22 Surprisingly, the availability of advanced technology seems not to improve the probabilities of rescuing patients. In the study by Wakeam et al.,23 it is concluded that availability of hospital clinical resources alone does not explain increased failure to rescuing rates.
The influence of hospital volume on the chance of rescuing complicated patients has been also discussed. According to some authors,24 patients operated at low volume centers for cardiac valve replacement are more likely to die if a complication occurs, but that finding is not a constant in all published papers. In the paper by Kurlansky et al.,25 the authors conclude that surgical outcomes were not associated with volume; instead they were directly correlated to compliance with evidence-based quality standards. In our study, the rate of successfully rescuing patients improved after implementing evidence-based guidelines for postoperative care, which is in consonance with Kurlansky et al. data.25
To note is the fact that in this study we have defined FTR as patient's death after major complications following Dindo et al.10 systematic classification of postoperative morbidity. Homogeneous definition of FTR is needed when benchmarking health care providers performance; interestingly, in the study by Farjah et al.,21 FTR rate increases considerably (with a median value of 10%) when only major complications are considered.
LimitationsThe most important limitation in our investigation is related to the long period of time included in the analysis. That could have influenced inadvertent changes in patient's management, maybe not at the Thoracic Department, since we have recorded all variations in perioperative practices, but in the practices of anesthesiologists or intensive care specialists we have not been aware of.
The retrospective nature or the process or re-coding complications is usually a source of errors. In this study, re-coding was accomplished by two investigators who have previously demonstrated a high agreement rate performing the task.26 Thus, we can assume a similar high rate of agreement happened in the current investigation.
CUSUM graphs were adjusted using the ESTS regression model for cardio-respiratory morbidity which is not specifically designed to be used with the Dindo et al. classification.10 For FTR, we used the ESTS’ EuroLung2 constructed for modeling hospital mortality, not for 30-day and, of course, not for FTR. We are not aware of any FTR-related European predictive models predicting FTR and, for that reason, we decided to adjust using EuroLung2. The high performance of the model (with a C-index over 0.8) may justify our decision.
ConclusionsIn our series, 22.5% of the patients suffering postoperative complications died and are recognized as FTR cases. FTR metric offers a complementary view to classical outcomes (morbidity and mortality) for quality assessment in Thoracic Surgery. On the other hand, CUSUM charts reveal specific points (cases 330 and 720) in which clinical measures were taken to improve patient management and its positive outcomes showing how compliance to evidence-based guidelines and not technology helped to improve the chances of rescuing patients after major postoperative complications.
Conflict of interestsThe authors declare no conflict of interest.
