CIBERESUCICOVID is a multicenter, observational, prospective/retrospective cohort study of patients with COVID-19 admitted to Spanish ICUs (Table 1). Our institution's internal review board (Comité Ètic d’Investigació Clínica, no. HCB/2020/0370) approved the study, and a member of the study team contacted local researchers, with participating hospitals obtaining local ethics committee approval as needed. Results were reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines.

Data collection started in May 2020, and we collected, anonymized, stored, and managed all data via the REDCap (Research Electronic Data Capture) tool, hosted at CIBER, Spain. Trained local researchers incorporated data from patients’ medical records into a separate database. We appointed a steering committee that met every week, and we created several work packages.

All consecutive patients admitted to participating ICUs from February 25th to February 28th, 2021, were enrolled if they fulfilled the following criteria:≥18 years old, ICU admission, and laboratory confirmed SARS-CoV-2 infection. Patients were excluded if they were admitted to an ICU for other reasons. After enrollment, we collected data on demographics, comorbidities, clinical symptoms, disease chronology, and treatment administered at hospital admission.

The site researchers subsequently collected data acquired at hospital admission, ICU admission, start of mechanical ventilation, 72-96 h after ICU admission, weaning off mechanical ventilation, ICU discharge, and hospital discharge. This included vital signs, respiratory support devices, adjunctive therapies, laboratory findings, and arterial blood gases. We studied hemodynamic parameters and organ dysfunction using the Sequential Organ Assessment Failure Score (SOFA) at ICU admission. We also collected data on pharmacological treatments and interventions administered at and during ICU admission until either ICU or hospital discharge or death. Mechanical ventilation settings were collected if used, and other pulmonary mechanical variables (e.g., driving pressure or static compliance of the respiratory system) were calculated. Impairment in oxygenation was analyzed using the PaO2/FiO2 ratio, and abnormalities of CO2 metabolism were studied using the ventilatory ratio (VR), a surrogate parameter of Vd/Vt.5 The worst event values were recorded preferentially.

The reported outcomes included in-hospital mortality, ICU mortality, 28-day mortality, 90-day mortality, duration of ventilation, and lengths of stay in ICU and hospital. ICU- and ventilator-free days were also computed as previously reported.6 All the pulmonary and extrapulmonary complications during ICU admission were also collected and included at hospital discharge.

General symptoms, respiratory symptoms, quality of life, and survival status were assessed at 3, 6, and 12 months after hospital discharge either during a home visit or phone call. Patients were asked about cardiovascular or infectious complications, emergency department visits, or hospital readmission. Post-COVID syndrome was defined according to the suggestion of the World Health Organization7 as the presence of symptoms such as fatigue, shortness of breath, and cognitive dysfunction that affect daily quality of life 3 months after probable or confirmed SARS-Cov2 infection, without being explained by other alternative diagnoses.

Airway function (spirometry, lung volume, and diffusing capacity) was measured according to the guidelines of the American Thoracic Society.8 Pulmonary parameters included forced vital capacity (FVC), forced expiratory volume in the first second (FEV1), the FEV1/FVC ratio, and lung diffusing capacity for carbon monoxide. The results were expressed as a percentage of the predicted value according to the European Community Lung Health Survey.9 A chest x-ray or computed tomography (CT) scan was also performed.

The 12-Item Short Form Survey (SF-12) was used to assess quality of life across eight health domains (general health, physical functioning, role physical, body pain, vitality, social functioning, role emotional, and mental health). This questionnaire has been validated in healthy populations, in several chronic diseases and conditions, and in acute respiratory distress syndrome (ARDS) survivors.10-12 The standardized SF-12 is described in means±standard deviation according to the normative standards established by Ware et al.,13 such that persons with a normal health-related quality of life have an average SF-12 score of 50±10.

We collected a 5mL blood sample in the first 48 h of ICU admission for COVID-19 infection, obtained 2mL of plasma from EDTA tubes after centrifugation at the recruiting center, and stored the sample at –80°C. Plasma samples were received centrally at the BioSepsis laboratory of Hospital Universitario Río Hortega de Valladolid/IBSAL, from where they were further split into three aliquots. The first was shipped to the laboratory of the National Center of Microbiology (Majadahonda, Spain) to profile anti-SARS-CoV-2 antibodies. The second aliquot was shipped to the IRB-Lleida (Spain) for microRNA profiling. The third remained at the BioSepsis group for biomarker profiling, antigenemia evaluation, and viral load quantification.

To identify biomarkers associated with an increased risk of mortality on ICU admission, we profiled 40 biomarkers using the Ella-SimplePlex™ system from Biotechnne (Lipocalin-2/NGAL, ICAM-1, Myeloperoxidase, VCAM-1/CD106, interleukin 1 beta, B7-H1/PD-L1, G-CSF, IFN-β, D-dimer, ferritin, E-selectin, RANTES, angiopoietin 2, GM-CSF, IL-7, IL-1ra, IL-6, IL-10, IL-17A, CXCL10/IP10, CCL2, granzyme B, IFN-γ, IL-12p70, IL-15, TNF-α, IL-2, IL-4, IFN-α, granzyme A, IL-8, endothelin-1, CD27, FAS, SPD, pentraxin 3, TREM-1, EGF, UPA-1, CTLA-4). This measured endothelial dysfunction, neutrophil degranulation, coagulation activation, neutrophil activation, T cell and monocyte chemotaxis, interferon response, and Th1, Th2, and TH17 response. Viral RNA was quantified in plasma by droplet-digital polymerase chain reaction (PCR) using the Bio-Rad SARS-CoV-2 ddPCR kit on a QX-200 ddPCR platform, according to manufacturer's specifications. The presence of N antigen of SARS-CoV-2 in plasma was assessed using the Panbio COVID-19 Ag Rapid Test Device (Abbott). We used a specific immunoassay developed by our team to quantify anti-SARS-CoV-2 S IgG and IgM antibodies.14

RNA isolation and microRNA (miRNA) quantification were conducted in the same laboratory using standardized protocols. Experienced staff blinded to patient data performed the miRNA quantification. Total RNA was isolated from frozen plasma using the miRNeasy Serum/Plasma Advanced kit (Qiagen, Germany) according to the manufacturer's instructions. For normalization, synthetic Caenorhabditis elegans miR-39-3p (cel-miR-39-3p) was added as an external reference miRNA. The mixture was supplemented with MS2 carrier RNA (Roche Diagnostics, Germany). The RNA Spike-In Kit (Qiagen, Germany) was added to monitor RNA isolation. RNA was eluted with RNAse-free water and stored at –80°C.

Reverse transcription (RT) was performed using the miRCURY LNA RT Kit (Qiagen, Germany). An additional spike-in UniSp6 (Qiagen, Germany) was added to monitor the RT reaction and complementary DNA was stored at 20°C. The miRNAs were analyzed using miRCURY LNA miRNA Custom Panels (384-well plates) (Qiagen, Germany). qPCR was carried out using the QuantStudioTM 7 Flex Real-Time PCR System (Applied Biosystems, USA). Synthetic UniSp3 was analyzed as an interplate calibrator and qPCR control. Amplification curves were evaluated using QuantStudio Software v1.3 (Thermo Fisher Scientific, USA).

To ensure optimal data quality, the spike-in RNA templates were first analyzed to monitor RNA extraction procedure uniformity and RT and PCR efficiency. Hemolysis contamination was excluded, as previously described. We considered miRNAs expressed at cycle quantification (Cq) values of<35. Relative quantification was performed using the 2-ΔCq method (ΔCq=CqmiRNA-Cqcel-miR-39-3p). Expression levels were log-transformed for statistical purposes. This protocol has been adapted for bronchial aspirates.15

CIBERESUCICOVID used REDCap as a single source of truth for data management. A second database was created to facilitate data analysis and exploitation, which was populated from REDCap through a periodic automated process. During this process, data were retrieved from REDCap forms in a specific order, units of measurement were unified, and variables were converted as necessary. Due to the number of patients and the amount of data, this process was parallelized for periodic completion. All data were stored in encrypted servers within the EU, and all data access was recorded and limited to authorized researchers. This architecture is further detailed elsewhere.16

Most analyses were automated to maximize scalability and reproducibility. The system could generate univariate analyses (e.g., basic statistics and statistical tests) and multivariate analyses (e.g., correlations and regressions), and survival curves (e.g., Kaplan-Meier), among others, by completing a short configuration (e.g., population restrictions and variables of interest). This automatization allowed us to iterate continuously in response to the many hypotheses proposed by medical experts in sub-studies, generating quick results with minimal overhead while ensuring reproducibility.

The wide scope and size of the CIBERES-UCI-COVID project enabled many derived sub-studies to emerge. To optimize time and resources, sub-studies went through several stages:

• 1.

  Definition of topic and responsible person.

• 2.

  Approval by project committee.

• 3.

  Definition of target population and variables of interest. This step produced a first set of tables with basic statistics and univariate test results that could be iterated upon based on the criteria of the responsible person.

• 4.

  Variable refinement (addition, deletion, modification) and multivariate analysis. This step was tailored for each sub-study and could include one or more analysis (e.g., population clustering, survival analysis, or propensity score analysis).

Valid scientific conclusions require the input of high-quality data with an excellent data quality assurance (DQA) process during all study phases (Table 2).

During data collection, we designed a parallelization strategy for data collection and cleaning. This used a custom-built DQA tool (based on the NAIHA medical device) that facilitated real-time communication using an application programming interface (API) with the study's electronic case report form (REDCap) (Fig. 1). This phase sought to monitor the data, generate reports, identify inconsistencies or missing data, and manage queries. The good design made it possible to develop the norms and rules to automate these tasks.

Figure 1.

The custom-built DQA tool (based on the NAIHA medical device) that facilitated real-time communication using an application programming interface (API) with the study's electronic case report form (REDCap).

(0.14MB).

During data analysis, we grouped all collected data by the working hypotheses or associated sub-studies for specific research. The data analysis module allowed the creation of filters to classify the ongoing sub-studies and to manage them independently. This way, we could manage the more than 5 million existing data points in an intuitive and efficient way, having only the necessary subjects and variables for each sub-study. This module personalized the query creation and sending, avoiding unnecessary requests being sent to the researchers.

The electronic case report form collected information on demographic data, comorbidities, and disease chronology, including treatments, interventions, outcomes, and complications during the hospital stay, with detailed clinical information on the most significant events, such as admission and discharge from the ICU or the start of mechanical ventilation. In total, the study included 55 Spanish ICUs in 13 autonomous communities, from which we included 8921 patients admitted consecutively to ICU with COVD-19 (Fig. 2). Of these, we obtained complete information on 6102 patients (70.3% men), exceeding the initial goal of 5000 patients, with a mean age of 63 (54-71) years. The most common comorbidities were arterial hypertension (50.4%) and diabetes mellitus (24.7%). On ICU admission, the APACHE-II and SOFA scores were 12 (9-16) and 5 (4-7), respectively. Also of note, 76% required invasive mechanical ventilation, increasing in-hospital mortality from 31% to 37% (Fig. 3). The average ICU stay was 13 (7-27) days and the average hospital stay was extended to 23 (14-40) days. Table 3 shows the demographic and clinical characteristics of these patients.

Figure 2.

Distribution of participating intensive care units over Spain's map.

(0.25MB).

Figure 3.

Sankey plot depicting distribution and outcomes of patients treated in the intensive care units according to the requirement of mechanical ventilation.

(0.15MB).

Of 3367 patients who survived hospitalization, 2382 completed a full 12-month follow-up period. During this period, 10% visited the emergency department and 1% died (17). At 12-months, spirometry data were available for 698 patients, lung diffusing capacity for carbon monoxide data for 581, and CT scan data for 602 (Fig. 4).

Figure 4.

Diagnostic tests distribution among 3-,6-, 12-months follow-up. Light colors display available patients with follow-up.

(0.12MB).

A total of 1338 samples were collected from 26 ICUs throughout Spain and screened for whether they met the inclusion and clinical data quality criteria. Finally, 1068 samples were processed, fractioned, and distributed according to the methods.

The CIBERESUCICOVID project pioneered in revealing the association of the plasma viral RNA load and antigenemia with critical illness and mortality.14,18 The project also demonstrated that the presence of viral RNA in plasma can be detected using standard real-time PCR methods.19 We described the association between plasma viral RNA load and the presence of a dysregulated host response to the infection, mimicking that occurring in sepsis.18 Furthermore, we described that low levels of anti-SARS-CoV-2 S antibodies were associated with mortality and the dissemination of viral components to the blood.14 These results led to two opinion papers,20,21 contributed to a review paper on COVID-19 pathogenesis,22 and to the submission of a patent on the use of antigenemia to predict mortality to the European Patent Office (EP20383140). Finally, using clinical follow-up data from survivors at 3-, 6-, and 12-month following hospital discharge, we can identify biomarkers linked to severity and outcomes in the acute phase, including the development of long COVID-19 symptoms in these patients. Research is ongoing.

In a preliminary study, we analyzed the circulating miRNA profile of patients admitted to ICU and in patients admitted to clinical wards without a requirement for critical care.23 We then quantified a panel of miRNAs previously associated with mechanistic pathways that could be altered in COVID-19 and demonstrated that specific circulating miRNA profiles are consistently associated with disease severity. Furthermore, we suggested that these represented a novel tool that could assist in the risk stratification of critically ill patients. A validation study in a large sample size is ongoing.

Since lung compartmentalization exists for multiple biological domains, an additional study was performed using respiratory samples.15 The same miRNA panel was quantified in bronchial aspirates from critically ill COVID-19 and non-COVID-19 patients, and we identified several individual miRNA ratios that predicted all-cause ICU mortality.15 In addition, we constructed a miRNA ratio-based prediction model for ICU mortality that showed an optimal discrimination value and outperformed the current best clinical predictor for ICU mortality.

Each sub-study followed a common iterative methodology agreed upon by the different parties. In each iteration, we first defined or revised the objectives, collected potentially relevant variables, and defined the sampling filters. Then, we selected the algorithms and methods for statistical analysis. Finally, we ran one or more experiments (based on different methods or their parameters) and generated detailed reports for experts to analyze and refine the objectives before starting a new iteration. Due to the iterative design, we prioritized having an automated process that could allow us to modify the least amount of code possible in each iteration (most commonly the revised set of relevant variables) and to generate interpretable detailed reports, regardless of the objectives, variables, and filters used in each sub-study. The processes of code validation, testing, database migration, and automated generation of different reports were integrated in a pipeline containing all stages needed to deliver a new version of the application. The automation of analysis allowed us to perform many sub-studies concurrently during the project. Nine of those were conducted through the automated analysis methodology, of which two have already been published17,24 and another two are under review with a journal. The remaining five are currently being drafted.

In the nine sub-studies, we produced 165.0 (12.0–370.0) univariate reports, 18.0 (6.0–30.0) multivariate analyses, and 5.0 (0.0–97.0) survival analyses, with a median of 28 propensity scores per sub-study. On average, it took 5.5 (3-11) months to obtain enough results to create a draft for a sub-study. This time included the evaluation and refinement of multiple medical hypothesis, as well as the production of the final figures and tables for publication.

We can summarize the results obtained by the DQA module as follows:

Unstructured text: These data include text with no structured format. Using a probability search algorithm, we converted the unstructured data to known tabulated data ready for processing by the data analytics module. We analyzed 30,566 unstructured texts from the 31,902 tabulated and validated texts (95.8%), discarding 542 and sending queries for 3.

Minimum data set (MDS) missing: This was the mandatory data to complete a sub-study. DQA analyzed 106,028 missing MDS and sent 623 queries, of which 326 were closed (52%).

Outliers: Data with values out of the statistical normal value range were queried in 4935 cases and resolved in 2427 (49%).

Inconsistent values: Conflicting values, like inconsistent dates or events. Inconsistent data could have more than one query because the incorrect value was not always clear. In total, 13,557 inconsistent values were queried, translating into 71,069 data queries. Currently, 19,329 data queries have been closed (27%).

We included 6102 consecutive patients admitted to 55 ICUs homogeneously distributed throughout Spain, with a general data manager (AC) coordinating the following steps:

• 1.

  Telephone contact of the principal investigator with the chief of the ICU.

• 2.

  After acceptance, they tried to obtain approval from the ethics committee.

• 3.

  The chief of the ICU appointed a person (or persons) to collect data for inclusion in the REDCAP database.

• 4.

  ISCIII contracting for each data entry investigator.

• 5.

  Some units preferred to be remunerated each time they collected data for a certain number of patients (e.g., each 100).

This process was certainly not easy and required diligence and a lot of energy. However, we learned the following lessons:

• 1.

  The Spanish Health System was not prepared for automatic data collection from ICUs.

• 2.

  ICUs use heterogeneous informatics systems that differ between regions and communities.

• 3.

  Current law does not allow for rapid ethic approval in emergency epidemics such as COVID-19.

These issues meant that we did not publish our first manuscript until October 2021,24 contrasting with other European countries.1,2 However, we spent time and effort to maximize the amount of data with strict quality control, something that the first rapid publications could not achieve.

Overall, the crude in-hospital mortality was 31% in our population, similar or lower than reported elsewhere.1 More than two thirds of the patients needed invasive mechanical ventilation, with mortality increasing to 37% in this population. The remaining patients were mostly treated with high-flow nasal cannula, or to a lesser extent, with bilevel or continuous positive airway pressure. Intubated patients were mostly maintained in the prone position. In a subset of patients (Hospital Arnau de Vilanova), late intubation (>48h after ICU admission) was associated with higher mortality compared to early intubation. We are currently analyzing this finding in the overall population.

As shown in previous publications, patients admitted to ICU developed acute kidney injury, nosocomial infections, or pulmonary complications2-4. Of particular significance was the rate of nosocomial bacterial pneumonia (30%), which was associated with several worse outcomes.25

One of our goals was to determine the prognostic factors of the target population. For that reason, we collected data at hospital admission, ICU admission, day 3 after ICU admission, extubation, ICU discharge, and hospital discharge. This allowed the examination of prognostic factors for more than 1000 patients requiring mechanical ventilation. Age and increases in the ventilatory ratio and creatinine from day 1 to day 3 independently and significantly predicted in-hospital mortality.24 The ventilatory ratio, an easily calculated bedside parameter that uses PaCO2 and minute ventilation, is a surrogate for measuring lung dead space.5 Studies are ongoing into the impact of delayed intubation, non-invasive ventilatory support, driving pressure, COPD, a very elderly population, and the effectivity of corticosteroids and remdesivir.

The present research also collected 1068 samples from patients admitted to ICU. Importantly, these samples were obtained in the first 48hours after ICU admission, centralized in the IBSAL laboratory, and distributed to other labs. We also made significant efforts to obtain as much clinical data as possible relating to the patients from whom we had access to blood samples. The related studies published to date14,18 show that the viral RNA load, the positivity of antigen tests in blood, and the presence of low amounts of, or absent, specific antibodies were independently associated with disease severity and/or 30-day mortality. These findings are of key importance and could very easily be adapted for implementation in our National Health System. Indeed, our results indicate that two types of COVID-19 patient arrive at ICUs: poor and good viral controllers. These exhibit different biological responses and clinical outcomes, and they are currently being characterized by our group in an ongoing analysis of the entire CIBERESUCICOVID cohort.

Blood testing also looked at miRNAs, for which we demonstrated individual and collective potential to improve risk stratification in the critically ill patient. The host miRNA profiles linked to the disease and to fatal outcomes may allow a deeper understanding of the mechanistic pathways implicated in critical COVID-19 illness. Currently, we are validating these findings in a large population comprising more than 500 samples from different Spanish hospitals and we are analyzing the association between the circulating miRNA profile, pulmonary function, and radiologic features in survivors of critical COVID-19 illness. This study aims to provide novel insights into the pulmonary sequelae of COVID-19.

We performed clinical follow-up studies in the overall population and in a cohort of patients from Hospital Arnau de Vilanova. In the overall cohort, age, ARDS, length of ICU stay, and ICU acquired nosocomial pneumonia were independently associated with long COVID syndrome at 3 months.26 In the Lleida cohort of patients with COVID-19 who were mechanically ventilated, 82% of patients who survived critical COVID-19 illness showed a lung diffusing capacity of less than 80% and abnormal chest CT scan in 70.2% at 3 months.27. In the overall population at 1 year follow-up, it was notable that the overall mortality of discharged patients was only 1%,17 a striking finding given that patients with community acquired pneumonia suffer mortality rates as high as 30%.28 Ongoing studies are considering the identification of clinical and biological predictors of long-term pulmonary sequelae.

We learned from our studies that patients discharged from ICU received care from several specialists and not intensivists. We have experiences difficulties in our efforts to contact these specialists, making it difficult to implement the follow-up protocol. This must be considered a limitation of the follow-up publications.

The integration of experts in medicine, data mining, and machine learning was a key challenge within the CIBERESUCICOVID project. On the one hand, we required a large number of medical hypotheses to be concurrently refined and tested, while on the other hand the different scientific practices of those disciplines demanded new definitions, formalizations, and automation of the research methodologies. Medical experts require certain statistical analyses for every data operation performed (e.g., univariate reports). This recurrent nature was recognized, and the process was automated, enabling certain sub-studies to produce in the order of thousands of statistical analyses. Data mining and machine learning methodologies, which may be unorthodox in medical practice, were suggested and integrated when possible (e.g., survival analysis or feature engineering). This enabled a distinct and heterogeneous approach to the problems and solutions posed by the pandemic, leading to a variety of sub-studies with differing complexities. At the same time, complex inconsistencies were automatically identified and classified with great precision. Synchronization with RedCap also worked efficiently to maintain real-time communication. However, we underestimated the many queries that would be generated, and to date, we have only debugged 65%, far from the projected 80%.