To evaluate the diagnostic accuracy of a non-invasive technology based on image processing for the identification of obstructive sleep apnea (OSA) and its severity at patients’ home.
MethodsObservational, prospective, diagnostic accuracy study to evaluate the degree of measure agreement between Sleepwise (SW), in-laboratory attended polysomnography (PSG) and a home sleep apnea test (HSAT). 38 consecutive subjects with suspected OSA referred as outpatients to the sleep unit were recruited from September 2016 to September 2017. All patients underwent in-laboratory attended PSG and image processing with SW simultaneously overnight. Subsequently, a HSAT and image processing with SW were performed simultaneously overnight at patients’ home, and the 2 nights after, patients underwent only image processing with SW consecutively.
ResultsIn-laboratory polysomnography and SW had a Lin's concordance correlation coefficient of 0.933 and a κ of 0.930. Between HSAT and SW the Lin's concordance correlation coefficient was 0.842 and a κ of 0.571. Agreement between two consecutive nights with SW recording showed a Lin's concordance correlation coefficient of 0.923 and a κ of 0. 837.
ConclusionsSW was highly accurate for non-invasive and automatic diagnosis of OSA in outpatients compared to standard methods for OSA diagnosis either in-laboratory attended PSG or HSAT. SW proved to be a technique with repeatable and concordant results on different nights for the same patient. We conclude SW is a non-invasive, easy-to-use, portable, effective and highly accurate system for the in-home diagnosis of OSA.
Evaluar la precisión diagnóstica de una tecnología no invasiva basada en el procesamiento de imágenes para la identificación de la apnea obstructiva del sueño (AOS) y su gravedad en el domicilio de los pacientes.
MétodosEstudio observacional, prospectivo y de precisión diagnóstica para evaluar el grado de concordancia entre las mediciones de Sleepwise (SW), polisomnografía (PSG) asistida en el laboratorio y poligrafía respiratoria domiciliaria (PRD). Se reclutaron 38 sujetos consecutivos con sospecha de AOS, referidos como pacientes ambulatorios a la unidad de sueño entre septiembre de 2016 y septiembre de 2017. Se les realizó a todos una PSG y procesamiento de imagen con SW durante la noche en el laboratorio. Posteriormente, se realizó una PRD y procesamiento de imágenes con SW de forma simultánea durante la noche en su domicilio, y las 2 noches siguientes los pacientes se sometieron solo a procesamiento de imágenes con SW.
ResultadosLa polisomnografía en el laboratorio y el SW mostraron un coeficiente de correlación de concordancia de Lin de 0,933 y un κ de 0,930. Entre la PRD y el SW el coeficiente de correlación de concordancia de Lin fue de 0,842 y un κ de 0,571. La concordancia entre las 2 noches consecutivas de grabación con el SW mostró un coeficiente de correlación de concordancia de Lin de 0,923 y un κ de 0,837.
ConclusionesEl SW mostró alta precisión para el diagnóstico no invasivo y automatizado de la AOS en pacientes ambulatorios en comparación con los métodos estándar para el diagnóstico de la AOS, ya sean la PSG asistida en el laboratorio o la PRD. El SW demostró ser una técnica con resultados reproducibles y concordantes en diferentes noches para el mismo paciente. Concluimos que la SW es un sistema no invasivo, fácil de usar, portátil, eficaz y altamente preciso para el diagnóstico domiciliario de la AOS.
In-laboratory polysomnography (PSG) is the gold standard for the diagnosis of obstructive sleep apnea (OSA).1 However, this technique is expensive and time-consuming.2 This condition added to the increasing prevalence of the disease explains the development of new systems for the diagnosis of OSA.3,4
Sleepwise (SW) is a non-invasive technology based on video image processing, by recording the patient's respiratory movements to transform them into a breathing signal that permits to determine episodes of hypopnea and apnea. Its technology is based on the principle that the volume of air that circulates into the lungs is proportional to the thoracic movement that a subject presents while breathing. Furthermore, according to the analysis of the body movements SW can also infer sleep/awake periods.
In a previous publication our group compared the diagnostic accuracy of SW and the in-laboratory polysomnography, concluding that Sleepwise determined the diagnosis and the severity of OSA with high reliability.5 We hypothesized that SW testing could also be performed at home. Therefore, the main objective of this study was to evaluate the diagnostic accuracy of SW compared with a home sleep apnea test (HSAT). As secondary objectives we checked SW reproducibility in different nights and also rated the easiness and comfort of its use.
Material and methodsPopulation of studyThis is an observational, prospective, diagnostic accuracy study that included consecutively 38 patients with suspected OSA who had been referred as outpatients to the Sleep Unit of the Hospital Universitari Germans Trias i Pujol (HUGTiP) from September 2016 to September 2017. Male and female patients over 18 years of age were required to sign an informed consent form to participate in the study. OSA suspicion was based on clinical criteria such as usual snoring, witnessed apnea and daytime sleepiness. Neurological and/or psychiatric disorders or any predictable difficulty with the understanding of HSAT or the video camera use were the only exclusion criteria.
The study was conducted according to the guidelines and principles of the Declaration of Helsinki and standard ethical conduct for research involving humans. The study also guarantees compliance with Organic Law 3/2018, of December 5, Protection of Personal Data and guarantee of digital rights (Spanish Government) and Regulation (EU) 2016/679 of the European Parliament and of the Council, of April 27 2016, with regard to the processing of personal data and on the free movement of such data. The Ethics Committees for Clinical Research of the participating center approved this study (REF. CEI: PI-15-142).
ProtocolSociodemographic (age, sex), anthropometric data (weight, height, neck, hip and waist circumference), sleep history and Epworth Sleepiness Scale (ESS) were collected for all participants.
All patients underwent overnight simultaneously PSG and SW recording at the sleep laboratory of the Sleep Unit of (HUGTiP). Subsequently, in a maximum time of 2 weeks after the in-laboratory test, patients slept at home simultaneously with a cardiorespiratory polygraph and SW recording for one night and the 2 consecutive following nights only with SW recording. PSG and cardiorespiratory polygraphy recordings were analyzed automatically and manually reviewed by the same certified sleep physician. The updated AASM 2007 classification was used to identify stages of sleep.6,7 The SW analysis was performed automatically.
PSG, HSAT and SW analyses were carried out independently and blindly by the same certified sleep physician.
The main outcome to compare both methods was the apnea–hypopnea index (AHI). Based on this, patients were classified as having mild OSA (5≥AHI<15), moderate OSA (15≥AHI<30) or severe OSA (AHI≥30), with an AHI of under 5 being deemed normal.4,6
Material- 1.
Sleepwise (SW)
SW is a non-invasive system for the diagnosis of OSA, which is able to detect respiratory events from the analysis of images provided by a conventional digital video camera equipped with infrared LEDs. This camera must be placed 60cm beside subject's bed focusing the thorax and the upper abdomen. SW technology is based in the principle that the volume of air that circulates into the lungs is proportional to the chest movement that a subject presents while breathing.
SW successively analyzes the images captured by the video camera and generates two types of signal. First, the respiratory movement signal that records subtle movements such as the thoracic oscillations that take place during breathing and can be used to infer respiratory flow and detect alterations therein. SW can detect respiratory movements independently of the position while sleeping or even if the patient is covered by a blanket.5 Second, the body movement signal that records movements involving a greater degree of motion, such as changes of body position. Based on a similar system to actimetry, this signal differentiates states of sleep/awake and can infer the subject's sleep time and number of awakenings.
To detect both respiratory events as well as the state of sleep/awake, SW uses numerical thresholds that were calculated empirically by an automatic learning system based on the results of polysomnography as the total number of events and sleep time.
For the in-laboratory tests, the technician set up both PSG and SW. For the in-home tests, participants were instructed on how to set up both HSAT and the video camera for SW recording by themselves. Participants were advised with verbal and written instructions and were given a brief demonstration. After using SW a self-questionnaire was passed to rate the comprehension and easiness of use, as well as the overall comfort of the system.
- 2.
Polysomnography (PSG)
For OSA diagnosis we used gold standard equipment in the form of a 32-channel E-Series polygraph (Compumedics Ltd.; Abbotsford, Victoria, Australia), to record electroencephalography, electroculography, electromyography and electrocardiography in accordance with AASM criteria (Type I study).6,7 We monitored respiratory flow by means of a thermistor and a nasal cannula. We recorded thorax and abdomen respiratory movements with two plethysmography bands, and oxygen saturation with a pulse oximeter. We used a video camera equipped with infrared LEDs to record each subject while sleeping.
We defined apnea as the complete cessation of respiratory flow for over 10s, and hypopnea as a reduction in respiratory flow lasting for over 10s and accompanied by oxygen desaturation of at least 3% and/or arousal according to AASM guidelines.6 We calculated the AHI as the quotient of the total number of apneas and hypopneas divided by the total number of hours of sleep determined by PSG.
- 3.
Home sleep apnea test (HSAT)
To perform the in-home studies we used Alice PDX (Philips Respironics, Murrysville, PA, USA), which is a portable monitor for the diagnosis of OSA (Type III study). It includes oxygen saturation (SpO2, finger probe, Oximetry board Nonin, Plymouth, MN, USA), pulse rate (from the oximeter probe), airflow (pressure-based airflow with snore detection through a nasal cannula and thermistor), thoracic and abdominal effort (inductance plethysmography), and body position.
Participants were advised with verbal and written instructions and were given a brief demonstration on how to set up the HSAT at home themselves.
Statistical analysisThe degree of agreement between procedures for the AHI was assessed by the Lin's concordance correlation coefficient, and the graphical representation of this agreement was described with the Bland–Altman method.
The degree of agreement between procedures for the definition of the severity of OSA was assessed by means of Cohen's kappa (κ), which improves upon simple percentage of agreement by taking into account the agreement occurring by chance. Kappa values range from 0 (when there is no agreement other than what would be expected by chance) to 1 (when the agreement is perfect). For this study, κ values greater than 0.81 were considered to be almost in perfect agreement; 0.61–0.80 were considered substantial; 0.41–0.60, moderate; 0.21–0.40, fair; and 0.00–0.20 were considered as slight agreement. Ninety-five percent confidence intervals of κ were also calculated.
All calculations were conducted in the version 15 of STATA using the agree and the cohenkap command.
ResultsThirty-eight patients were included in the study and their clinical data are shown in Tables 1 and 2. Agreements between procedures are shown in Table 3. The graphical representation of this agreement, described with the Bland–Altman method can be seen in Figs. 1–3.
Characteristics of the 38 patients included in the study.
Characteristics | |
---|---|
Age (years), mean (SD) | 46.8 (11.4) |
Female, n (%) | 24 (63) |
Smoker, n (%) | 9 (23.1) |
Epworth, mean (SD) | 6.8 (5.6) |
Height (cm), mean (SD) | 170 (11) |
Weight (kg), mean (SD) | 89.6 (23.6) |
BMI (kg/m2), mean (SD) | 30.7 (7.6) |
Neck (cm), mean (SD) | 39.3 (7.5) |
Wrist, mean (SD) | 102.0 (18.4) |
Hip, mean (SD) | 107.6 (12.9) |
Hypertension, n (%) | 14 (36.8) |
Diabetes, n (%) | 5 (13.2) |
Dyslipidemia, n (%) | 7 (18.4) |
Ischemic cardiopathy, n (%) | 2 (5.3) |
AHI, mean (SD) | 36.8 (27.4) |
Degree of agreement between procedures assessed by the Lin's concordance correlation coefficient and Cohen's κ.
Comparisons | Apnea–hypopnea index | OSA severity | |
---|---|---|---|
CCC | kappa | 95%CI | |
PSG-SW0 | 0.933 | 0.930 | 0.867–0.993 |
HSAT-SW1 | 0.917 | 0.793 | 0.713–0.872 |
SW2-SW3 | 0.923 | 0.837 | 0.729–0.944 |
CCC: concordance correlation coefficient.
There was almost perfect agreement between PSG and SW, with a Lin's concordance correlation coefficient of 0.933 and a κ of 0.930 (95% CI 0.867–0.993).
Between HSAT and SW, agreement was lower, with a Lin's concordance correlation coefficient of 0.917 and a κ of 0.793 (95% CI 0.713–0.872).
Agreement between two different SW procedures was again almost perfect, with a Lin's concordance correlation coefficient of 0.923 and a κ of 0.837 (95% CI 0.729–0.944).
None of the participants reported difficulties in understanding the functioning of SW. The device was considered easy-to-use and comfortable. All the recordings had an adequate quality for their proper analysis.
DiscussionIn this present study we found a very high correlation between SW and the gold-standard methods for OSA diagnosis both in-laboratory and in-home, as well as a robust reproducibility of the test. These findings are in line with our previous study, in which we described a good agreement between PSG and SW.5 Therefore, we propose SW as an innovative and reliable system for the in-home diagnosis of OSA.
In-laboratory, technician-attended PSG monitoring at least sleep stages and respiration, is accepted as the gold standard for the diagnosis of OSA.8 However, this procedure requires technical expertise, is labor-intensive and time-consuming and together with the high prevalence of OSA and the great demand of examinations results in a problem of long waiting lists. Reliable diagnostic systems that short cut the conventional test are thus a must. Portable monitors are alternative approaches to diagnosis providing an equivalent diagnosis to in-laboratory PSG.3 SW is not just a portable in-home device but more importantly it is also a non-invasive system. SW does not use any sensor attached to patient's body which implies several advantages compared to other systems for the diagnosis of OSA. First, it is more comfortable for the subject as SW records patient's sleep without any nuisance and then the quality of sleep is therefore probably improved. SW can detect respiratory movements independently of the position while sleeping or even if the patient is covered by a blanket. Second, it is easier to use as patients can be able to set up the system themselves at home with brief instructions. All participants in our study rated the instructions to set up SW as easy or very easy and expressed their satisfaction to the system comfort. The fact we excluded participants presenting difficulties for the understanding of the set-up of the system could be a selection bias. However, these patients would not have likely been candidates for a HSAT either.
One of the limitations of HSAT is the absence of sleep staging evaluation that provides a denominator for the AHI. This type of portable monitor provides breathing events quantified per hour of monitoring time as a respiratory disturbance index (RDI) that can underestimate the severity of OSA. Regarding this point, SW offers an undeniable advantage over the rest of the currently available portable diagnostic systems. One SW's limitation is the absence of cardiorespiratory data registration, such as oxygen saturation, pulse or cardiac rhythm that helps physicians to determine the severity of the apneas and hypopneas and their systemic repercussion. It is technically possible to add the information in question, but that would lead to a drawback of the system ceasing to be noninvasive. However, future SW versions would incorporate these variables obtained in a non-invasive and friendly system, such as it has been agreed with experts as fundamental for the evaluation of OSA.
Although it is difficult to compare two different procedures by simply comparing the kappa index, our findings suggest that SW could be more reliable than HSAT to define OSA's severity. The explanation for this fact is that SW calculates breathing events based on the time of sleep and not the monitoring time.
However, we found 3 cases with substantial differences between AHI value determined by PSG and SW. These concrete patients were severe OSA with a BMI over 30 and this findings concur with our previous study.5 SW and PSG presented larger differences in the AHI value in obese patients with severe disease although patients were properly classified as severe OSA by both techniques, thus without implications in the therapeutic decision in any case.
HSAT use is accepted for the diagnosis of OSA in uncomplicated adult patients presenting with signs and symptoms that indicate an increased risk of moderate to severe OSA. On the contrary, polysomnography should be elected in patients with significant cardiorespiratory disease, neuromuscular disease with respiratory impairment, suspicion of hypoventilation, opioid medication use, history of stroke, or severe insomnia.4 Participants of this study did not presented any major comorbidity to contraindicate a HSAT. In the future it would be interesting to compare SW agreement with PSG in complicated adult patients.
The AASM recommends that the raw data from the HSAT devices must be reviewed and interpreted by a physician who is either board certified in sleep medicine or overseen by a board certified sleep medicine physician.9 Although in this study we provided the results of automatic analysis by SW, the system also allows to review the raw data manually.
Even though reproducibility of the two SW procedures was considered to be almost perfect, kappa values were slightly lower than that of the comparison between PSG and SW. The reason for this could be the fact that those procedures were performed in two different nights, with presumably different AHI due to a night-to-night variability. This error could be solved in the future by performing a study comparing two simultaneous SW recordings overnight on the same individual. This would be interesting to evaluate the system repeatability which is currently assumed to be high according to our results.
Other authors have developed various techniques based on motion analysis for diagnosis of OSA, but there are significant differences when compared to SW from the video analysis of breathing to the detection of respiratory events.10–13 All this studies were performed in an experimental, restricted environment with control subjects, but never tested in real life or compared to other diagnostic techniques. A strength of our study lies in demonstrating the utility and applicability of SW as an in-home diagnosis system of OSAS while previous studies have always been developed in a laboratory environment.
The benefits presented by SW lead us to hypothesize that it could be a useful tool to study pediatric patients. We have analyzed some pediatric patients but preliminary results have not reached the same level of perfect agreement found in the study with adults. The most probable explanation is that children's respiratory mechanics and body size are different from adults and thus the same algorithm cannot be applied. Further studies are needed to adapt the algorithm and improve the effectiveness of the device in this concrete population group.
Economic analyses have compared the cost-effectiveness of management pathways that incorporate diagnostic strategies using HSAT or PSG.14–16 Contrary to what one might think, all have concluded that PSG is the preferred diagnostic strategy from an economic perspective for adults suspected to have moderate to severe OSA. The reason for that is the favorable cost-effectiveness of OSA treatment in this group of patients particularly when longer time horizons are considered. HSAT could lead to increase false negatives, and so leave patients untreated, or increase false positives and consequently treat unnecessarily patients. The impact of these errors can be magnified when extrapolated over long time horizons. The HomePAP study concluded that for payers, a home-based diagnostic pathway for OSA with robust patient support incurs fewer costs than a laboratory-based pathway, but for providers, costs are comparable if not higher, resulting in a negative operating margin.17 In that sense, our previous results comparing SW to PSG demonstrated a sensitivity of a 100%, specificity of 87%, a positive predictive value of 97% and a negative predictive value of 100% for the diagnosis of OSA.5 According to that, we believe SW would be a more cost-effective system compared to available HSAT adding also that not consumable material is required and all the automatic analysis is performed digitally. However, this is a hypothesis to be confirmed in further specific cost-effective studies.
ConclusionIn conclusion, SW was found to be highly accurate for non-invasive and automatic diagnosis of OSA in outpatients when compared to standard methods for OSA diagnosis both in-laboratory and in-home. SW proved to be a technique with repeatable and concordant results on different nights for the same patient and resulted easy to set and very comfortable for the patients. According to these results, we conclude SW is a remarkable non-invasive, easy-to-use, portable, effective and highly accurate option for OSA diagnosis at patients’ home.
FundingThe research reported in this publication was supported by a grant from the Spanish Society of Pulmonary Medicine and Thoracic Surgery (SEPAR) with code 030/2015. Esteve-Teijin Healthcare, S.L collaborated disinterestedly with our project instructing the participants in the set-up of both the respiratory polygraph and the camera for the Sleepwise recording for the in-home tests.
AuthorshipAll authors have contributed to the conception and design of the study; analysis and interpretation of data; and revising the article critically for important intellectual content. IAV and CF collected the data. IGO and JA performed the statistical analysis and interpreted the results. AMF wrote the manuscript. All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors read and approved the final manuscript.
Conflict of interestsMAC is Chief Executive Officer (CEO) of Tarsio Medics, S.L, Malgrat de Mar (Barcelona), Spain. AMF, JRM and JA are minority shareholders of Tarsio Medics, S.L, Malgrat de Mar (Barcelona), Spain since July 2017 and serve as clinical advisors for this company. The terms of this arrangement have been reviewed and approved by the Fundació Institut d’Investigació en Ciències de la Salut Germans Trias i Pujol (IGTP) in accordance with its policy on objectivity in research. IGO, IV, CF and AR declare that there is no conflict of interests regarding the publication of this paper.
Appreciation is expressed to Maria José Masdeu, MD, PhD and Ferran Barbé, MD, PhD for their critical comments on the manuscript. We are also greatly thankful to Richard O’Hegarty, PhD for his help with the manuscript correction. The authors thank the sleep laboratory nurses, including Francisco Javier Abril, Juan Treviño and Marta Finistrosa, for their help with the realization of the sleep tests. The authors take into great consideration the collaboration of Esteve-Teijin Healthcare, S.L and its physiotherapists Monica Sanz and Susana Pou for instructing the participants in the set-up of the respiratory polygraph and the camera for the in-home registers.
This study was performed at Hospital Universitari Germans Trias i Pujol (HUGTiP), Badalona (Barcelona), Spain.