The quality of care received by patients is a basic element of modern medicine. Medical residents or interns are essential within the healthcare system, but their lack of experience can raise concerns about the quality of care given. A registrar or specialist has greater knowledge and skills, while a resident has greater motivation and enthusiasm. The aim of training programs is to prepare residents to provide high quality care. This requires close supervision that seems to be lacking, with the consequent impact on both healthcare quality and academic results.
The so-called “July effect” refers to the diminished quality of care during the summer months when resident physicians switch over. The results of studies analysing this effect vary widely, but the loss of efficacy during these months does seem to be real.
Pulmonology is one of the medical specialties that generates the least demand for internships and residencies, but it is impossible to determine if this affects the quality of care. The high prevalence of respiratory diseases and the latest diagnostic and therapeutic advances may mean that this situation will change in coming years.
La calidad de la atención que recibe el paciente es un aspecto fundamental de la medicina actual. Los residentes son esenciales en la organización sanitaria, pero su falta de experiencia produce la preocupación de que descienda la calidad asistencial entregada. Un adjunto tiene mayores conocimientos y habilidades, un residente mayor motivación y entusiasmo. El objetivo de los programas formativos es preparar a los residentes para proporcionar unos cuidados de alta calidad. Es fundamental para ello su supervisión, que parece ser inadecuada e influye tanto en la calidad asistencial como en sus resultados académicos.
La disminución de la calidad asistencial en los meses de recambio de residentes es el llamado «efecto julio», y aunque los estudios que analizan dicho efecto tengan resultados heterogéneos, la efectividad parece verse realmente afectada en estos meses.
Neumología es una de las últimas especialidades médicas en adjudicar sus plazas MIR, sin que podamos precisar si eso influye en la calidad asistencial. La alta prevalencia de las enfermedades respiratorias y los últimos avances diagnósticos y terapéuticos podrían producir un cambio de esta situación en los próximos años.
Quality is understood as “the degree to which health services […] increase the likelihood of desired health outcomes and are consistent with current professional knowledge”.1 The quality of care in health services has become one of the key issues in modern medicine. Innumerable problems have arisen in recent decades, stemming from the increasing complexity of health systems, development of medical technology, greater demands by patients and their families, and spiraling costs. These problems require effective solutions which can guarantee patients the right treatment, at the right time, and within a framework of equality and respect for their rights and values2; in other words, the establishment of procedures that guarantee quality care.
Society must accept that junior doctors need to acquire experience that they can later benefit from; however, it must also insist that the healthcare system and the individuals responsible for this training ensure that risks are minimized. The National Council of Medical Specialties Guidelines for Specialized Training3 recommends that tutors protect patients by supervising their students while gradually increasing their responsibilities.
It is universally accepted that safety and high quality patient care are only possible if doctors are well prepared to meet these demands during their residency training. This is why training programs are aimed at preparing residents to provide high quality care. Clinicians worldwide involved in residency programs are focused on improving education by modernizing competency-based training and assessing the quality of resident training using accredited standards.
The following article is based on presentations made at the Sixth National Forum of Young Pulmonologists, held during the 47th National SEPAR conference (Spanish Society of Pulmonology and Thoracic Surgery) in Bilbao in 2014, organized by the Healthcare Quality Committee. It attempts to answer some key questions that will encourage readers to reflect on the need to detect, solve and avoid mistakes, and to improve quality of care. To that end, the following points will be discussed: (1) Is it better to be seen by a specialist or a resident? (2) The influence of resident supervision; (3) The July effect; and (4) Why is respiratory medicine is such low demand as a specialty?
Is it Better to be Seen by a Specialist or a Resident?Most patients prefer to be seen by a specialist rather than a resident. In a survey administered to patients in an emergency department, 79.5% of respondents expected to be assessed by a specialist, regardless of the acuity of symptoms or potential cost-savings.4 We can speculate that this is probably due to the specialist's years of experience in clinical practice, greater knowledge and training in clinical skills, dexterity in performing procedures, certainty in delivering diagnosis, and efficacy in implementing treatments. Specialists also bear legal responsibility for the consequences of their actions. However, residents contribute other elements that should not be underestimated: young doctors are generally more highly motivated than specialists; their enthusiasm and desire to learn mean that they are often more aware of the latest developments in their field; they show a greater capacity for work, enduring 24-h on-call shifts, sometimes with no time off (either because of hospital/department policy or their own interest in continuing to learn), and being considered “cheap labor”, they are often required to perform tasks not theoretically assigned to them (due to departmental needs).
Previous studies have attempted to relate the grade obtained in qualifying examinations with the quality of care that these doctors might provide in their future practice. Wenghofer et al.5 selected 208 doctors taking QE1 (medical knowledge) and QE2 (clinical skills) exams in Canada between 1993 and 1996, and followed them for 7–10 years of clinical practice. They observed that those in the bottom quartiles in both tests were ultimately at greater risk of failing their quality of care evaluation. The authors concluded, therefore, that these scores could be significant predictors of future quality-of-care problems.
Evaluating clinical performance is a basic element in training. This initially posed a challenge, as historically the evaluation of a doctor's performance was implicit, non-standardized, and based on subjective opinions. The need to change this system led to reforms in medical training that introduced new systems for evaluating skills and activity. Miller et al.6 conducted a systematic review of studies evaluating the impact of workplace-based assessment on doctor's education and performance. They observed that both aspects improved when assessments were received from multiple sources, and that changes in clinical practice were more likely to occur when the assessment was credible or helped the subjects to identify their weak points. This has also been demonstrated in more specific contexts: determining the quality of care in emergency departments can potentially be used in the assessment of training programs,7 and linking competency-based learning objectives in critical care units8 with clinical outcomes was shown to improve both resident education and patient care.
One of the most extensive systematic reviews9 carried out so far encompassed all articles published from December 2004 to February 2011, and evaluated the impact of residency training on patient health outcomes. Ninety-seven articles were included; of these, 43 evaluated the care outcomes achieved by residents compared to the “gold standard” care provided by specialists. In 31 of these 43 articles, there were no statistically significant differences between specialists and residents in a wide variety of outcomes such as length of hospital stay, morbidity and mortality. However, in 12 of these 31, surgery times were longer in interventions undertaken by residents. Nine of the 43 studies showed poorer outcomes in resident care (mainly in terms of morbidity, and increased length of hospital stay in patients seen by unsupervised residents), and 3 studies showed poorer outcomes in specialist care, probably due to selection bias.
In 16 of these studies, the need for supervision was underlined, as outcomes were worse among unsupervised residents.
Another issue concerns possible differences between the first and subsequent years of residency. A study conducted in residents from a respiratory care department did not find differences in monthly weaning rates or mortality rate between first-year and more senior residents. The authors concluded that, in a well organized setting, the level (experience) of the resident does not significantly affect patient outcomes.10
Influence of Resident SupervisionResident supervision and duty hours have been a matter of public importance since 1987, when a law was passed in New York regulating, for the first time, the number of hours that residents should work each week.11 These limits formed the basis of the Accreditation Council for Graduate Medical Education (ACGME) recommendations on supervision and duty hours, published in 2003 and in 2010. In 2008, the American Institute of Medicine (IOM) recommended increased resident supervision in training programs, including night and weekend shifts.12
In Spain, the MIR (resident training program) contract13 states that residents must learn by working under the supervision of a tutor and a teaching committee. These latter must periodically evaluate the resident's performance and draw up a record of the objectives achieved, and on this basis give him or her the autonomy needed to act as a specialist. However, some studies show that resident supervision is inadequate, and that up to 37.2% of residents are dissatisfied with the supervision received.14
Direct observation by the supervising specialist is essential for evaluating a resident's training and assessing their clinical and communication skills. However, the best tools for assessing these skills are unclear. A systematic review of articles describing evaluation tools15 included 10672 citations from June 1965 to March 2009, and analyzed 55 tools in the 85 articles that met inclusion criteria (mainly scoring systems for rating skills in completing medical records, ability to perform a physical examination, communication skills, etc.) They observed that greater supervision could be associated with better patient care and more rapid acquisition of clinical skills, as it involves detailed assessment and feedback from the specialist to the resident. However, this is either rarely or inadequately followed in clinical practice, and further studies are needed to analyze these supervision tools more thoroughly.
The issue of whether resident supervision affects outcomes is hotly debated. Most studies support the theory that supervision positively affects outcomes.9,16–18 Supervision can impact patients and improve the resident's academic standing.
Few studies show that the presence of the specialist has no effect on outcomes, and that patient morbidity and mortality are not affected by residents working under different levels of supervision.19 In general, research shows that a greater number of incorrect diagnoses, medical procedure-related complications, non-compliance with protocols, and major changes in the patient's therapeutic plan occur when residents are not supervised. However, most studies conclude that failures and changes are related to the resident's degree of training20; in this respect, more senior residents have the same outcomes regardless of the degree of supervision.9,21,22
Scant research has been conducted on the effect of supervision on the resident's academic results. The aspects evaluated are the ability to make diagnoses and perform procedures, the perception of autonomy and overall satisfaction.
In 2012, Farnan et al. carried out a systematic literature review on the supervision of residents between 1966 and 2010.20 They included 24 studies, most of which showed that direct supervision of residents improved patient- and education-related outcomes. The presence of the specialist evaluating the resident usually changed the overall assessment of the diagnosis or treatment (residents had a tendency to overestimate the severity of the disease); furthermore, most studies showed that these errors and changes were directly related with the resident's level of experience.
The supervisor's position is another widely discussed aspect. It is agreed that a qualified supervisor is not necessarily an individual who has completed their post-graduate training; on the contrary, the supervisor can be a more senior resident, provided they have more training than their trainee. Thus, some studies show that the errors and changes needed in the diagnoses and work plans of unsupervised residents depends on the year of residency and degree of training. This means that more senior residents make fewer mistakes, and therefore fewer changes are needed; moreover, they are able to supervise more junior residents.20
For these reasons, in 2011 the ACGME recommended that residents should be supervised by an immediately senior resident or by a specialist, who should be present and available to assist immediately.12
The “July Effect”The “July effect” is a term used in the United States to describe the impact on the quality of patient care caused by the changeover of residents at academic year-end. This situation has also been described in the United Kingdom as the “August killing season”, as this is when this switch occurs.23–25
At the start of the academic year, as experienced residents leave and new residents arrive, the collective experience of hospital residents will obviously be significantly impoverished with respect to the previous month. This is thought to be the reason why more mistakes are made in the first few months of the academic year.26 Previous studies have evaluated morbidity and mortality, surgical outcomes, length of hospital stay and increased costs in this period compared with non-teaching hospitals or with other times of the year, with heterogeneous results.26,27 However, no studies have been conducted in Spain on this so-called effect, or on the consequences of the annual resident changeover.
In one study, the most methodologically sound studies conducted between January 1989 and July 2010 on the issue of the July effect were reviewed.23 The studies were classified according to methodological quality, sample size and outcomes. A total of 39 articles were eventually reviewed after applying the inclusion criteria: 27 analyzed mortality, 23 morbidity, 19 efficiency (length of stay, duration of procedure, hospital costs) and 6 medical error outcomes.24,28
Only 6 of the 27 studies on mortality outcomes showed increased mortality. Four of these had a sufficiently large sample size to detect statistically significant differences. The higher quality studies showed an association between resident changeover and mortality.
Four of the 23 studies reporting morbidity outcomes found increased morbidity. The morbidities evaluated in these articles were: intraoperative complications, undesirable anesthesia-related events, iatrogenic pneumothorax, and the need to refer patients to a nursing home. However, the authors cautioned that only the study reporting anesthesia-related events was of good methodological quality.
Seven out of 19 studies investigating efficiency found shortcomings in this aspect. The length of hospital stay, hospital costs, and length of surgery were some of the variables evaluated. A greater loss of efficiency was found in higher-quality studies.
Finally, studies reporting on medical error outcomes analyzed both errors in medical treatments and unscheduled return visits to the emergency department, but these were of very poor quality, with inconsistent results.
The review, therefore, concluded that mortality and efficiency in hospitals in this period of the academic year are indeed affected by the changeover phenomenon. However, because of their heterogeneity, no firm conclusions on the degree of risk, effect on morbidity, or rates of medical error can be drawn from the studies.
Proposed solutions include improving resident supervision, previous training in simulation workshops, overlapping residents for at least one week so that new trainees can become familiar with the institution and care procedures under the supervision of experienced residents, graduated clinical responsibilities (especially at night) and/or staggered start dates so that the changeover is more gradual. Other points to be addressed would include giving new residents the confidence to speak up, admit errors, teaching them to identify mistakes and to be accountable for their actions.23,28
Why is Respiratory Medicine in Such Low Demand as a Specialty?Medical specialisation in Spain is ruled by a system widely known as MIR, which stands for “Médico Interno Residente”, literally ‘resident medical intern’. Annualy, 47 postgraduate training programmes in different specialties are offered, imposing limits on the number of physicians in each programme. The order for choosing the training programmes in determined by the ranking of the applicants. In the 2011 MIR cycle, for example, the Ministry of Health authorised 6881 positions in 560 training centres throughout the country.
Although, as discussed above, scores in general examinations affect subsequent outcomes in clinical practice, there are no studies on the association of MIR examination score with quality in later practice. Recent years have seen a gradual decline in the choice of respiratory medicine when allocating MIR places. Although there are no studies relating the score obtained in the MIR examination with subsequent quality of care, it is hard not to wonder why respiratory medicine residency programs are taken up by students with such high scores in the MIR exam.
The specialty currently occupies one of the lower positions among the medical specialty options. Only nephrology, hematology, critical care medicine and allergology offer a similar, or even fewer, number of residency programs, and more are offered in psychiatry, internal medicine and general practice. In the last 2 MIR examinations, in 2013 and 2014, the last program in respiratory medicine was awarded to applicants ranked at position 4684 and 4703, respectively, among the total number of applicants in Spain. In a well-known Spanish blog dedicated to analyzing the MIR population, a table of trends in demand for specialties shows that respiratory medicine has been ranked around 28 for the last 10 years, far below specialties such as cardiology, gastrointestinal medicine and neurology.29
There may be several reasons behind this: (1) Superficial knowledge of respiratory disease during university studies could result in little interest in pursuing the specialty; therefore, students should be encouraged to have direct contact with clinic practice and respiratory techniques. (2) The lower social prestige of respiratory medicine compared to other medical specialties. This phenomenon could be a result of lack of awareness of the specialty and the importance of respiratory diseases in the general public. This was shown in a survey conducted in 1994 by SEPAR, where 90% of respondents did not know the meaning of the word “pulmonologist”,30 which means that we sometimes have to present ourselves as “the lung doctor”. According to a more recent survey focusing on chronic obstructive pulmonary disease (COPD) in Spain, 88.8% of the respondents did not know what the disease is, or what the acronym COPD stands for.31 (3) Difficulty in finding a job when the residency period ends, particularly in private practice.32 This may depend on the lack of demand among the general public for respiratory care, lack of awareness of the specialty, and lack of access to the resources needed to make a complete assessment of the respiratory patient (imaging tests, lung function tests, etc.). (4) The fact that the respiratory medicine residency programs still last for 4 years is a disadvantage compared to other specialties such as cardiology or internal medicine. Nowadays, sub-specialization, development of new techniques, and the need to broaden knowledge in research require the training period to be extended. It is the responsibility of both pulmonologists and the scientific societies that represent them to raise the profile of this specialty in order to counterbalance this effect. Efforts in this regard should be aimed at both the general public and at public health authorities and hospital administrations.30
Respiratory medicine has strengths upon which to build to help the specialty grow, one of which is the high prevalence of respiratory diseases. These are the third most common cause of death in Spain, excluding lung cancer (included under malignant diseases), which has the highest mortality rate.33 This has led to the steady growth of this specialty, in which advances in the development of diagnostic and therapeutic techniques, introduction of new bronchodilator drugs, new inhaled antibiotics and new anti-fibrotic treatments, have helped it adapt to the needs of a constantly evolving society. Another important aspect of respiratory medicine is the creation and development of scientific societies for respiratory diseases (SEPAR and other regional societies). These bodies have done much to raise the profile of respiratory medicine, particularly through their platform for continued training and continuing professional development. In the field of research, respiratory medicine has achieved major objectives in recent years: projects have been funded by the Spanish Health Research Fund (FIS), the specialty is part of the Spanish national research, development and innovation (R&D+I) plan, and the Biomedical Research Center Network for Respiratory Diseases and Integrated Research Programs has been developed. All this will no doubt attract future residents, and hopefully in the coming years the profile for awarding MIR respiratory medicine residency programs will change.
ConclusionsResidents are characterized by their enthusiasm, motivation, their close involvement with their patients, and their eagerness to improve and expand their knowledge by studying the latest guidelines and updates. Quality of care will be similar to that of a specialist, provided that this is directly and properly supervised by a responsible specialist or resident experienced in the area. The care delivered by first-year residents with respect to more senior residents does not significantly affect patient outcomes, provided that each resident is allocated work commensurate with their level of experience.
The staff changeover period in hospitals during academic year-end (“July effect”) can undermine care quality. This must be taken into account and measures implemented to prevent its consequences. For this purpose, new methodologically sound studies are required to analyze the impact of this effect in Spain.
Respiratory medicine currently ranks among the least in-demand medical specialties in the allocation of MIR places; the causes are varied, although it should be stressed that respiratory medicine has strengths that have led to the growth of the specialty in recent years.
Conflict of InterestsThe authors declare that they have no conflict of interests.
The authors would like to thank the SEPAR Care Quality Committee for organizing the Young Pulmonologists Forum at the national congress; the Editorial Committee of Archivos de Bronconeumologíafor considering its content to be of interest; and all the residents and specialists who attended the presentations and debates in the aforementioned forum.
Please cite this article as: Llopis Pastor E, Pérez Guzmán E, Ávila Martínez R, Villena Garrido MV, Sobradillo Ecenarro P. Impacto de la calidad de la atención por parte de los residentes. Arch Bronconeumol. 2015;51:515–519.
