External Validation of The Society of Thoracic Surgeons General Thoracic Surgery Database

doi:10.1016/j.athoracsur.2013.04.124

The Annals of Thoracic Surgery

Volume 96, Issue 5, November 2013, Pages 1734-1739

Presented at the Forty-ninth Annual Meeting of The Society of Thoracic Surgeons, Los Angeles, CA, Jan 26–30, 2013.

https://doi.org/10.1016/j.athoracsur.2013.04.124 Get rights and content

Background

The Society of Thoracic Surgeons (STS) General Thoracic Surgery Database (GTSD) reports outstanding results for lung and esophageal cancer resection. However, a major weakness of the GTSD has been the lack of validation of this voluntary registry. The purpose of this study was to perform an external, independent audit to assess the accuracy of the data collection process and the quality of the database.

Methods

An independent firm was contracted to audit 5% of sites randomly selected from the GTDB in 2011. Audits were performed remotely to maximize the number of audits performed and reduce cost. Auditors compared lobectomy cases submitted to the GTSD with the hospital operative logs to evaluate completeness of the data. In addition, 20 lobectomy records from each site were audited in detail. Agreement rates were calculated for 32 individual data elements, 7 data categories pertaining to patient status or care delivery, and an overall agreement rate for each site. Six process variables were also evaluated to assess best practice for data collection and submission.

Results

Ten sites were audited from the 222 participants. Comparison of the 559 submitted lobectomy cases with operative logs from each site identified 28 omissions, a 94.6% agreement rate (discrepancies/site range, 2 to 27). Importantly, cases not submitted had no mortality or major morbidity, indicating a lack of purposeful omission. The aggregate agreement rates for all categories were greater than 90%. The overall data accuracy was 94.9%.

Conclusions

External audits of the GTSD validate the accuracy and completeness of the data. Careful examination of unreported cases demonstrated no purposeful omission or gaming. Although these preliminary results are quite good, it is imperative that the audit process is refined and continues to expand along with the GTSD to insure reliability of the database. The audit results are currently being incorporated into educational and quality improvement processes to add further value.

Section snippets

Material and Methods

The STS GTSD was initiated in 2003 and the first validation of the GTSD was conducted in 2010. The audit process was patterned after that previously performed to validate the STS Adult Cardiac Surgery Database, with specific parameters developed within the GTSD and audit taskforces 7, 8. The first two audits have focused on lobectomy. Thirty-two specific individual data elements within seven categories of variables were selected based on defined quality care measures, specific data elements

Results

A total of 559 lobectomy cases submitted by the 20 audited sites to the database were compared with operative logs from each site. The number of cases for comparison at each site ranged from 20 to 196, with the number of discrepancies per site ranging from 2 to 27. A total of 28 omissions were identified, resulting in a 94.9% overall agreement rate. Importantly, those cases not submitted had no mortality or major morbidity indicating a lack of purposeful omission.

The data collection methods

Comment

The STS surgeons’ audit of the GTSD was designed and conducted to evaluate the accuracy, consistency, and completeness of data collection and ultimately to validate the quality of the data. The overall high agreement rates reported for all variables analyzed in the audit process demonstrate that the data contained in the GTSD are both complete and accurate.

The process of comparing hospital surgery logs of eligible lobectomy cases to those submitted to the database was included in the audit to

References (8)

M.J. Mack et al.
Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes?
J Thorac Cardiovasc Surg
(2005)
D.F. Torchiana et al.
Use of administrative data for clinical quality measurement
J Thorac Cardiovasc Surg
(2005)
B.D. Kozower et al.
STS database risk models: predictors of mortality and major morbidity for lung cancer resection
Ann Thorac Surg
(2010)
C.D. Wright et al.
Predictors of major morbidity and mortality after esophagectomy for esophageal cancer: a Society of Thoracic Surgeons General Thoracic Surgery database risk adjustment model
J Thorac Cardiovasc Surg
(2009)

There are more references available in the full text version of this article.

Cited by (38)

Pre-COVID-19 National Mortality Trends in Open and Video-Assisted Lobectomy for Non–Small Cell Lung Cancer
2022, Journal of Surgical Research
Citation Excerpt :
The NIS also does not contain details on patient presentation, cancer pathology, surgeon experience, and details of surgical procedure, which would have been important, as these have previously been identified as risk factors for adverse outcome. However, our dataset may provide more information about real-world practice as opposed to databases, such as the GTSD, which is less generalizable because of the voluntary participation of institutions included in it, making it prone to selection bias.16,17 In this nationally representative, multi-institutional study based on insurance claims data, we demonstrate that utilization of VATS lobectomies has steadily increased over time, more so among small hospitals compared with large hospitals.
In the current era of episode-based hospital reimbursements, it is important to determine the impact of hospital size on contemporary national trends in surgical technique and outcomes of lobectomy.
Patients aged >18 y undergoing open and video-assisted thoracoscopic surgery (VATS) lobectomy from 2008 to 2014 were identified using insurance claims data from the National Inpatient Sample. The impact of hospital size on surgical approach and outcomes for both open and VATS lobectomy were analyzed.
Over the 7-y period, 202,668 lobectomies were performed nationally, including 71,638 VATS and 131,030 open. Although the overall number of lobectomies decreased (30,058 in 2008 versus 27,340 in 2014, P < 0.01), the proportion of VATS lobectomies increased (24.0% versus 46.9%), and open lobectomies decreased (76.0% versus 53.0%, all P < 0.01). When stratified by hospital size, small hospitals had a significant increase in the proportion of open lobectomies (6.4%-12.2%; P = 0.01) and trend toward increased number of VATS lobectomies (2.7%-12.2%). Annual mortality rates for VATS (range: 1.0%-1.9%) and open (range: 1.9%-2.4%) lobectomy did not significantly differ over time (all P > 0.05) but did decrease among small hospitals (4.1%-1.3% and 5.1%-1.1% for VATS and open, respectively; both P < 0.05). After adjusting for confounders, hospital bed size was not a predictor of in-hospital mortality.
Utilization of VATS lobectomies has increased over time, more so among small hospitals. Mortality rates for open lobectomy remain consistently higher than VATS lobectomy (range 0.4%-1.4%) but did not significantly differ over time. This data can help benchmark hospital performance in the future.
National Trends and Outcomes of Segmentectomy in the Society of Thoracic Surgeons Database
2022, Annals of Thoracic Surgery
Segmentectomy is gaining popularity as a parenchyma-sparing alternative for anatomic lung resection. This study sought to investigate temporal changes in patient selection, case volume, and outcomes for segmentectomy using the Society of Thoracic Surgeons (STS) National Database.
The STS General Thoracic Database was queried for patients who had undergone segmentectomy as the primary procedure between 2002 and 2018. The American College of Surgeons Oncology Group definition of high-risk patients on the basis of pulmonary function and major cardiovascular comorbidities was applied. Annual trends of case volume, patient risk profile, surgical indication, approach, and outcomes were analyzed.
A total of 10 629 patients were analyzed from 310 contributing centers. The annual segmentectomy volume more than doubled from <4 per center in 2009 to 8.6 per center by 2017. Lung cancer was the most common indication (70.1%), followed by benign disease (15.6%) and metastatic tumors (14.3%). Although the operative indication remained constant, the subset of high-risk patients (24.5%) decreased gradually over time (slope, −0.6% per year; P = .001). After 2012, segmentectomies were most commonly performed minimally invasively (video-assisted thoracoscopic surgery, 58.3%; robotic surgery, 19.4%), with a steadily declining use of thoracotomy (overall, 22.3%; slope, −2.4%/y; P = .001). Overall complication rates decreased over the study period from 41.7% to 26.1% (slope, −0.57%/y; P = .001). The overall major complication rate was 4.6% (range, 2.0%-7.1%), 30-day mortality was 1.0% (range, 0.7%-5.0%), and both have been trending downward since 2009 (P = .01).
Segmentectomies are increasingly performed nationally, with a steady decline in the subset of high-risk patients over time. Complication rates have decreased significantly, parallel to the increasing use of minimally invasive techniques by video-assisted thoracoscopic surgery and robotic surgery.
The influence of tobacco load versus smoking status on outcomes following lobectomy for lung cancer in a statewide quality collaborative
2021, Journal of Thoracic and Cardiovascular Surgery
Collaborative quality consortia can facilitate implementation of quality measures arising from clinical databases. Our statewide general thoracic surgery (GTS) collaborative investigated the influences of cigarette smoking status on mortality and major morbidity following lobectomy for lung cancer.
Society of Thoracic Surgeons General Thoracic Surgery Database records were identified from 14 institutions participating in a statewide thoracic surgical quality collaborative between 2012 and 2017. We excluded patients with nonelective procedures, stage 0 tumors, American Society of Anesthesiologists class VI disease, and missing clinical characteristics. Outcomes analysis included the combined mortality and major postoperative morbidity rates and the influence of patient characteristics, including smoking status, on composite rate and on postoperative complications.
The study cohort included 2267 patient records for analysis. Overall combined mortality and major morbidity rate was 10.2% (n = 231). Postoperative 30-day mortality was 1.5%, and major morbidity 9.6%. Significant predictors of the combined outcome included male sex (P = .004), body mass index (P < .001), Zubrod score (P = .02), smoking pack-years (P = .03), and thoracotomy (P < .001). Higher American Society of Anesthesiologists disease class and advanced tumor stage were marginally associated with worse combined outcome (P = .06). Smoking status; that is, current, past (no smoking within 30 days), or never smoked, was not associated with worse combined outcome (P = .56) and had no significant influence on major complications.
Smoking status was not associated with worse outcomes; however, smoking dose (pack-years) was associated with worse combined mortality and major morbidity. A statewide quality collaborative provides constructive feedback for participating institutions and surgeons, promoting quality improvement in perioperative patient care strategies and improved outcomes.
Costs Associated With Lobectomy for Lung Cancer: An Analysis Merging STS and Medicare Data
2021, Annals of Thoracic Surgery
Costs related to care of patients who undergo lobectomy for lung cancer may vary depending on patient, disease, and treating facility characteristics. We aimed to identify underlying case mix factors that contribute to variability of 90-day costs of lobectomy for early-stage lung cancer.
The Society of Thoracic Surgeons General Thoracic Surgery Database was queried for lobectomy for clinical stage I lung cancer (2008-2013). Demographics, clinical outcomes, and 90-day episode-of-care costs across all care settings were analyzed for patients successfully linked to Medicare data. Hospital costs were estimated from charges using cost-to-charge ratios. Comprehensive regression models were created to identify impact of preoperative patient factors and hospital characteristics on costs, and to delineate additive costs due to perioperative outcomes and complications.
The mean 90-day cost for lobectomy was $45,080 ± $38,239. Variables associated with significant additive costs were age greater than or equal to 75 years, American Society of Anesthesiologists classification III or IV, forced expiratory volume in 1 second less than 80% predicted, body mass index less than 18.5 or greater than 35, current or past smoker, cerebrovascular disease, chronic kidney disease, impaired functional status, open thoracotomy, prolonged operative time, government hospitals, metropolitan setting, and geographic location. Patients with 1 or more postoperative complication resulted in an overall mean added cost of $27,259. Added costs increased with the number of complications; isolated recurrent laryngeal nerve paresis ($3,911) and respiratory failure ($35,011) were associated with the least and most additive cost, respectively.
Lobectomy is associated with substantial variability of episode-of-care costs. Variability is driven by patient demographic and clinical factors, hospital characteristics, and the occurrence and severity of complications.
Equivalent Survival Between Lobectomy and Segmentectomy for Clinical Stage IA Lung Cancer
2020, Annals of Thoracic Surgery
The oncologic efficacy of segmentectomy is controversial. We compared long-term survival in clinical stage IA (T1N0) Medicare patients undergoing lobectomy and segmentectomy in The Society of Thoracic Surgeons database.
The Society of Thoracic Surgeons General Thoracic Surgery Database was linked to Medicare data in 14,286 lung cancer patients who underwent segmentectomy (n = 1654) or lobectomy (n = 12,632) for clinical stage IA disease from 2002 to 2015. Cox regression was used to create a long-term survival model. Patients were then propensity matched on demographic and clinical variables to derive matched pairs.
In Cox modeling segmentectomy was associated with survival similar to lobectomy in the entire cohort (hazard ratio, 1.04; 95% confidence interval, 0.89-1.20; P = .64) and in the matched subcohort. A subanalysis restricted to the 2009 to 2015 population (n = 11,811), when T1a tumors were specified and positron emission tomography results and mediastinal staging procedures were accurately recorded in the database, also showed that segmentectomy and lobectomy continue to have similar survival (hazard ratio, 1.00; 95% confidence interval, 0.87-1.16). Subanalysis of the pathologic N0 patients demonstrated the same results.
Lobectomy and segmentectomy for early-stage lung cancer are equally effective treatments with similar survival. Surgeons from The Society of Thoracic Surgeons database appear to be selecting patients appropriately for sublobar procedures.
Spanish Video-Assisted Thoracic Surgery Group: Method, Auditing, and Initial Results From a National Prospective Cohort of Patients Receiving Anatomical Lung Resections
2020, Archivos de Bronconeumologia
Our study sought to know the current implementation of video-assisted thoracoscopic surgery (VATS) for anatomical lung resections in Spain. We present our initial results and describe the auditing systems developed by the Spanish VATS Group (GEVATS).
We conducted a prospective multicentre cohort study that included patients receiving anatomical lung resections between 12/20/2016 and 03/20/2018. The main quality controls consisted of determining the recruitment rate of each centre and the accuracy of the perioperative data collected based on six key variables. The implications of a low recruitment rate were analysed for “90-day mortality” and “Grade IIIb-V complications”.
The series was composed of 3533 cases (1917 VATS; 54.3%) across 33 departments. The centres’ median recruitment rate was 99% (25–75th:76–100%), with an overall recruitment rate of 83% and a data accuracy of 98%. We were unable to demonstrate a significant association between the recruitment rate and the risk of morbidity/mortality, but a trend was found in the unadjusted analysis for those centres with recruitment rates lower than 80% (centres with 95–100% rates as reference): grade IIIb-V OR = 0.61 (p = 0.081), 90-day mortality OR = 0.46 (p = 0.051).
More than half of the anatomical lung resections in Spain are performed via VATS. According to our results, the centre's recruitment rate and its potential implications due to selection bias, should deserve further attention by the main voluntary multicentre studies of our speciality. The high representativeness as well as the reliability of the GEVATS data constitute a fundamental point of departure for this nationwide cohort.
Nuestro estudio buscó conocer el grado de implementación actual de la cirugía toracoscópica asistida por video (VATS, por sus siglas en inglés) para las resecciones pulmonares anatómicas en España. Presentamos nuestros resultados iniciales y describimos los sistemas de auditoría desarrollados por el grupo español de VATS (GEVATS).
Realizamos un estudio de cohortes prospectivo multicéntrico que incluyó pacientes que fueron tratados con resecciones pulmonares anatómicas entre el 20/12/2016 y el 20/03/2018. Los controles de calidad principales consistieron en determinar la tasa de reclutamiento de cada centro y la precisión de los datos perioperatorios recolectados en base a seis variables clave. Se analizaron las implicaciones de una baja tasa de reclutamiento para “mortalidad a los 90 días” y “complicaciones de grado IIIb-V”.
La serie estaba compuesta por 3533 casos (1917 VATS; 54,3%) en 33 servicios. La mediana de la tasa de reclutamiento de los centros fue del 99% (p25-p75: 76-100%), con una tasa de reclutamiento global del 83% y una precisión de los datos del 98%. No pudimos demostrar una asociación significativa entre la tasa de reclutamiento y el riesgo de morbi-mortalidad, pero se encontró una tendencia en el análisis no ajustado para aquellos centros con tasas de reclutamiento inferiores al 80% (usando los centros con tasas de 95-100% como referencia): OR = 0,61 para el grado IIIb-V (p = 0,081), OR = 0,46 para la mortalidad a los 90 días (p = 0,051).
Más de la mitad de las resecciones pulmonares anatómicas en España se realizan a través de VATS. Según nuestros resultados, la tasa de reclutamiento del centro y sus posibles implicaciones debido al sesgo de selección, deberían recibir más atención por parte de los principales estudios multicéntricos voluntarios de nuestra especialidad. La alta representatividad y la confiabilidad de los datos de GEVATS constituyen un punto de partida fundamental para esta cohorte nacional.

View all citing articles on Scopus

View full text

Original articleGeneral thoracicExternal Validation of The Society of Thoracic Surgeons General Thoracic Surgery Database

Background

Methods

Results

Conclusions

Section snippets

Material and Methods

Results

Comment

J Thorac Cardiovasc Surg

J Thorac Cardiovasc Surg

Ann Thorac Surg

J Thorac Cardiovasc Surg

Original article
General thoracic
External Validation of The Society of Thoracic Surgeons General Thoracic Surgery Database