Surviving a Site Visit by the RRC
(Adapted from a presentation by Fred Crawford, M.D., RRC for Thoracic Surgery member, TSDA Meeting, January, 1998)
Every Program Director is interested in knowing how to survive a site visit and subsequent review by the RRC without receiving that dreaded letter suggesting that his/her program is anything but perfect.
I will review some of the common pitfalls and errors made by program directors (including myself) in dealing with this process. This discussion may seem incredibly basic to some of you, and in fact, it is not rocket science.
The first step in the review process occurs when you receive "The Letter" from the RRC stating that you will be site visited on a particular date by a member of the field staff. The first mistake program directors commonly make is to wait until they receive the notification letter to begin to address issues that will be raised at the time of the site visit and the subsequent review. In order to avoid this, you must make yourself intimately familiar with the Program Requirements for Residency Education in Thoracic Surgery (see Tab L, previous section). Like any other "game", this one has a rule book, and the Program Requirements explain how this game is played. A program director recently responded to an unfavorable action of the RRC by stating "I would like to point out that these items [program deficiencies] have been news to us since we have never received these guidelines of the Essentials of Accredited Residencies which are mentioned in your letter." It is every program director's obligation to obtain and be familiar with the Program Requirements. It is basically too late to put an organized curriculum into effect, be able to prove that you have consistently evaluated your residents in writing and they have done the same for you and your faculty, and deal with consistently inadequate operative experience in one or more categories when the site visit is scheduled to occur within six months. I am really talking to those of you who do not have a site visit already scheduled, and I plead with you to go back home, read these Program Requirements and be sure that your program complies with them. Many of the Program Requirements deal with documentation. Again, this is not rocket science, but attention to detail will allow you to document in advance those things which are clearly spelled out in the Program Requirements.
The second mistake is to tell your secretary to file the letter from the RRC and remind you the week before the information is due. The report you will prepare for the RRC is a comprehensive document that requires a fair amount of work to complete properly. Do not put off the data gathering until the last minute. The members of the RRC are individuals like you, and some are sitting in this room today. We are busy with all of the same things that you are. We end up reading and reviewing the information submitted by program directors in bits of time that we snatch from something else, and nothing is more irritating than trying to read through information provided by a program director that is obviously hastily put together, disorganized, clearly inaccurate or incomplete. It almost inevitably biases the reviewer as to the quality of that program. So when you get that letter announcing your scheduled site visit, go to work, delegate parts of it to your fellow faculty members, and follow the instructions precisely. Provide the information that is requested, and make it as concise but complete as possible. Pay particular attention to those forms dealing with operative experience. Make sure that the numbers for operative experience by residents match the institutional operative case totals.
For example, if the operative experience form for all of the institutions in your program indicates that there were 15 esophagectomies done by all hospitals in the past year, and 14 of those were done by residents, it becomes a little unbelievable when the two residents in the program indicate that during the same time period, they each did 14! You should probably not delegate compilation of these data to a novice secretary or an administrative assistant, as operative case lists are probably the most inaccurately submitted data received by the RRC for review.
Now I would like to go through the Program Requirements for Residency Education in Thoracic Surgery. I am not going to review each of the Requirements in detail, but I am going to point out those that are most often cited in adverse action letters to program directors. The order in which I discuss these will not be an indication of their importance nor of the order of frequency with which they are cited but simply the order in which they appear in the document.
I.B.1 The director and teaching staff of the program must prepare and comply with written educational goals for the program. This statement obviously implies that there are indeed written educational goals for the program, and you may be asked to supply these for review when the site visitor comes.
II.B.1. The resident must have the opportunity under supervision to provide operative management including the selection and timing of operative intervention and the selection of appropriate operative procedures. This requirement clearly implies that the resident will participate in the preoperative evaluation of the patient and simply not show up in the operating room and construct bypasses to vessels as indicated by the attending. Residents will be quizzed about this in their individual interview with the site visitor. In the Program Information Forms the program director is asked to document how residents progress through the program from doing simple cases to the more complex and how the resident is provided graded responsibility.
III.B.2. Facilities must be appropriate and sufficient for the effective accomplishment of the educational mission of each program . . . There must be adequate space for patient care, program support and research, trained medical and paramedical personnel, laboratory and imaging support, and medical library resources. Implicit in this is that the residents will have some office space of their own, and that there should be an immediately available library. They should have computer access, if not in their own office, then from the divisional offices, etc.
The next several items deal with responsibilities of the Program Director.
IV.A.2.a. The responsibilities of the program director include preparation of a written statement outlining the educational goals of the program with respect to knowledge, skills and other attributes of residents at each level of training and for each major rotation or other program assignment. This statement must be distributed to residents and members of the teaching staff and be readily available for review. Almost certainly the site visitor will ask to see this document and will note in his review whether it exists or not. This is currently considered an important issue.
IV.A.2.d. The responsibilities of the program director include "the supervision of residents through explicit WRITTEN descriptions of supervisory lines of responsibility for the care of patient." This requires that a chain of command exists, that it is documented in writing, and that the residents understand it.
IV.A.2.g. The responsibilities of the program director include "ensuring assignment of reasonable in-hospital duty hours so that residents are not required regularly to perform excessively prolonged periods of duty. It is desirable that the residents' work schedule be designed so that on average, excluding exceptional patient care needs, residents have at least 1 day out of 7 free of routine responsibilities and are on call in the hospital no more often than every third night. Since training in thoracic surgery is at senior levels, the ratio of hours worked and on-call time varies considerably and necessitates flexibility." Also fitting under that category is V.F. which states, "The educational mission of each program must not be compromised by an excessive reliance on residents to fulfill institutional service obligations, and
IV.E.2. a portion of which states "duty assignments must not regularly be of such excessive length and frequency that they cause undue fatigue and sleep deprivation". Each of these deal with the struggle between service responsibilities and educational activities. The fact that the same theme appears in 3 separate areas of the Program Requirements underscores its importance. The Thoracic Surgery RRC is generally understanding and reasonably flexible about these requirements, but if the site visit should turn up data to suggest that residents are being utilized primarily for service responsibilities instead of education, and if their duty hours are truly excessive, then this will be scrutinized carefully. Most often, this information surfaces during the time of interview with the resident staff and may serve as a focal point of complaint when residents are unhappy with their program.
IV.A.2.h. The responsibilities of the program director include notification of the Executive Secretary of the RRC in writing of any program change that may significantly alter the educational experience for the resident. The issue here is prior approval of the RRC which is required for any significant changes such as rotations to other institutions, length of the program, number of residents, etc.
IV.B.3. There should be at least 1 designated cardiothoracic faculty member responsible for coordinating multidisciplinary clinical conferences and organizing instruction and research in general thoracic surgery. If your program does not have a designated individual responsible for instruction in general thoracic surgery, the program will be cited.
IV.B.5 The teaching staff must be organized and have regular documented meetings to review program goals and objectives . . . at least 1 resident representative should participate in these reviews. At the time of the site visit, you will probably be asked to provide minutes of your faculty meetings. Resident participation must be documented.
IV.D. A portion of which states - Adequate documentation of scholarly activity on the part of the program director and the teaching staff at the sponsoring and integrated institutions must be submitted at the time of the program review. We have reviewed programs in which there have been no publications or research grants for years and in which the faculty have not even attended major meetings for several years. The focus of these programs is relatively obvious.
V.A. Guidelines for minimal operative experience of each resident include:
a. an annual average of 125 major operations
b. an adequate distribution of categories and complexity of procedures such that each resident is ensured a balanced and equivalent operative experience.
It is essential that your program have a way of tracking on an ongoing basis resident operative experience, and this experience should be reviewed by the program director and the faculty at specific intervals. It will not help a great deal if the first time you review a resident's operative experience is when you go to sign off for him on the American Board of Thoracic Surgery forms and discover that he has done only 6 congenital heart procedures and 1 esophageal resection. As mentioned previously, the residents' operative experience must be accurately documented and must be consistent with the institutional experience.
V.A.2. Operative experience should meet the following criteria:
a. The resident participated in the diagnosis, preoperative planning, and selection of the operation for the patient.
b. The resident performed those technical manipulations that constituted the essential parts of the patient's operation.
c. The resident was substantially involved in postoperative care.
d. The resident was supervised by responsible teaching staff.
This requirement explains clearly when a resident may take credit for having performed an operative procedure. Program directors and faculty should be sure that the residents understand these requirements, and there must be agreement at the time of each operation, whether or not the resident is allowed to claim it as "his or her" case,. As you know, data submitted to the American Board of Thoracic Surgery is now considered final at the time of first submission, and individuals turned down for the certifying examination because of inadequate experience in one or more areas may no longer go back and "recount" their cases, claiming that they did not understand the requirements for counting cases, and for example, that they really did 10 esophagectomies and not 2. One of the most frequent citations is consistent inability of a program director to provide adequate operative experience in one or more categories on a repeated basis. The two most frequent deficient areas are congenital heart surgery and esophageal surgery. Suffice it to say that if your program has over the years provided inadequate experience in more than one category, and particularly if one or more of your residents has not been allowed to take the American Board of Thoracic Surgery examination for this reason, your program is in significant danger of losing RRC accreditation.
V.C. The objectives of each rotation should be well defined in the curriculum. It is likely that the site visitor will ask you to show him a document that outlines the experience you expect the resident to gain from each individual rotation.
V.E. The program director is responsible for providing separate and regularly scheduled teaching conferences, mortality and morbidity conferences, rounds, and other educational activities in which both the thoracic surgery teaching staff and the residents attend and participate . . . .Records of attendance must be kept and available for review by the site visitor. The residents will be asked at the time of the site visit about these conferences, how often they are held, their quality, and whether or not patient care responsibilities permit them time to regularly attend. Program directors will likely be asked to document the fact that these conferences were held and attended. In general, the RRC looks unfavorably upon a haphazard series of lectures and now is looking specifically to see if programs are following the proposed curriculum outlined by the TSDA.
V.G. Outpatient activities constitute an essential component for providing adequate experience in continuity of patient care . . . To participate in posthospital care, the resident has the responsibility for seeing the patient personally in an outpatient setting and/or, as a minimum in some cases, consulting with the attending surgeon regarding the follow-up care rendered to the patient in the doctor's office. The policies and procedures governing prehospital and posthospital involvement of the residents must be documented. This requirement presents a problem for most of us. It is not expected that every single patient must be seen in follow-up by the resident. However, there must be some system in place which permits the residents to see in follow-up a significant portion of the patients that they have operated upon, either in a residents' clinic or in the attending physician's clinic at the time the patient returns.
VI.A. The educational effectiveness of a program must be evaluated in a systemic manner . . . Written evaluations by residents of the teaching staff . . . should be utilized in this process.
VI.B. Regular evaluation of residents' knowledge, skills, and overall performance . . should be documented. The program director shall: (1) at least semiannually evaluate the knowledge, skills and professional growth of the residents using appropriate criteria and procedures; (2) communicate each evaluation to the resident in a timely manner; (4) maintain a permanent record of evaluation of each resident and have it accessible to the resident and other authorized personnel.
VI.C. A written final evaluation for each resident who completes the program must be provided. These last 2 requirements deal with evaluation of a resident and faculty at periodic intervals and documentation in writing of their evaluations. It is particularly important to have in the resident folder a final written evaluation of the resident at the time he/she completes the program. Most certainly, this will be asked for by the reviewer.
VI.D. One measure of the quality of a program may be the percentage of program graduates who take the certification examination of the ABTS and their performance on that examination. Clearly, significant failure on the part of your residents to pass the ABTS certifying examination will reflect unfavorably on your program.
There are numerous other things listed in the Program Requirements that you must deal with, document, and either provide information about on the Program Information Forms or have available to the site visitor. Those that I have discussed are those most commonly cited at the time of review by the RRC. Failure to comply with these in particular will almost certainly result in a citation, and if there are numerous areas of noncompliance, may result in an adverse action by the RRC.
I would suggest that you involve your faculty and particularly your residents in the preparation of the program information forms and in the overall preparation for the site visit. As I have outlined, many of the areas that will be scrutinized are known in advance, and it is wise to prepare your faculty and residents so that they may discuss them intelligently with the site visitor. If there are clear cut disagreements about various aspects of the program either among the various faculty or between residents and faculty, it is far better to have discussed these than have them surface at the time of the site visit.
I hope that this presentation has been useful and not too simplistic. Most of it has been learned from years of experience, not all of it pleasant, as well as my recent time on the RRC.
