The Annual Review of Competency Progression is an annual formal review to decide whether trainees have attained the necessary core curriculum competences in their current training year, in order to progress to the next stage of training. A formal panel including a training programme director is convened for this.
The ARCPs will be done electronically using trainees’ Intercollegiate Surgical Curriculum Programme (ISCP) ePortfolios as the primary source of information. Trainees will be asked to have these up to date by a specific date before the day their ARCP takes place.
It is the responsibility of the trainee to ensure that by their ARCP and Interim Review:
- sufficient numbers of workplace-based assessments (WBA) are completed and that the correct proportion are undertaken with consultants
- their e-logbook is linked in and available on their ISCP portfolio for the panel to review
- their exam passes, publications, audits, research and teaching/courses attended are recorded in the evidence section of the ISCP portfolio
Trainees will only be invited to a face to face review if the panel have sufficient concerns about their progress at this stage.
Please see the full ARCP guidance and checklists on the curricula and programme for documentation requirements in the evidence section for the ISCP portfolio. Additional requirement guidance has been provided below:
The School of Surgery expects that a minimum number of the following assessment tools are carried out by the time of their interim review:
– Six CBD (three to be done with a consultant)
– Six Mini-CEX (three to be done with a consultant)
– Six DOPS (CT1) (three to be done with a consultant) or PBAs (CT2) (three to be done with a consultant)
By the time of ARCP, trainees must have completed a minimum number of the following assessment tools and provide evidence of:
– 12 CBD (six to be done with a consultant)
– 12 Mini-CEX (six to be done with a consultant)
– 12 DOPS (CT1 six to be done with a consultant)
– 12 DOPS/PBAs (CT2 – six to be done with a consultant, PBAs only count with a consultant)
– One Mini-PAT (compulsory in CT1 and CT2)
– Six Teaching Days
– Leadership Topic
– Non-Technical Skills for Surgeons (NOTSS) Assessment
– Form R Parts A & B
The actual number is predicated on training delivering curriculum requirements, to enable the trainee to gain sufficient depth of experience. The trainee’s Assigned Educational Supervisor (AES) has a key role in judging whether the trainee requires more than the minimum number.
During training, WBA is part of a learning cycle, providing the basis for on-going dialogue and regular feedback. Following an assessment, the trainee should reflect on feedback and augment practice with further study in a safe environment e.g. skills lab or coaching. Emerging problems should be followed by targeted corrective action and interim assessments.
One point worth emphasising relates to assessments of technical procedures (i.e. Direct Observation of Procedural skills (DOPS) and Procedure-based Assessments (PBA). During training, a trainee will be learning a range of technical procedures simultaneously, and the indicative number of assessments is for each one of these technical procedures, so that progression can be gauged for all the procedures that the trainee is seeking to learn at that point in time.
A range of assessments indicates whether the trainee is making satisfactory progress in terms of meeting overall educational goals. When a trainee has demonstrated achievement of one level of curriculum competencies, it is expected that they will show they are maintaining that level of competence while being assessed on the next level of competencies.
Assessment for learning should be seen as an on-going process. When we discuss frequency below we refer to the minimum standards expected for trainers and trainees to achieve. The minimum standard is defined as that which is required to provide a reliable assessment. Ideally, assessment for learning should be occurring as frequently as learning opportunities arise. Every encounter between trainer and trainee should be seen as a debriefing and feedback opportunity i.e. regular feedback becomes the norm.
Frequency of assessments
Spreading assessments, e.g. at least one per week with a mix of type throughout the placement, optimises the effectiveness of WBA and improves reliability. Trainees should request assessment for any new task that they undertake as soon as possible. On-going constructive feedback is conducive to reflection and helps keep progress on track; it also provides the AES with information about the trainee’s rate of learning and developmental needs.
Observing the trainee with several patients is desirable from an educational perspective because different patients in different settings require different skills from trainees and this significantly broadens the range and richness of feedback they receive. The only implication of an individual assessment that is unsatisfactory is that it should be repeated at a later date, ideally by another assessor.
Similarly, trainees should be evaluated by different members of their team who have their own expertise, strengths and perspectives. Feedback from several different assessors will benefit trainees as it improves reliability. Trainees should actively seek feedback from the whole team, including patients and their carers. The philosophy of what can be learned to become better should be at the core of professionalism.
The primary function of the rating scale is to inform the trainee and the trainer about what needs to be learned; therefore rating trainees accurately is critical to the success of WBA. It is essential that everyone who is responsible for assessing trainees is trained to ensure they understand how to use the rating scales so that ratings can be applied consistently.
The standard against which the trainee should be rated differs according to the assessment tool.
Most of the assessments compare the trainee’s performance against the curriculum standard of a doctor at the end of that particular stage of training, rather than at the level of the assessor’s personal expectation of the trainee at that point in time. Assuming that WBA appropriately challenges the trainee, the norm for a trainee at the start of a stage would be a rating of below expectations (rating 1-2). A rating of meets expectations (rating 4) should be achieved by the end of the training stage. Ratings of above expectations (rating 5-6) indicate that the trainee can manage all the competencies, including complications without any supervision. It would be rare for trainees to attain this level in the early years.
PBAs rate trainees’ performance against the standard required for CCT, both for the individual items and the global summary. A satisfactory standard for individual items in a PBA will often be achieved at an early stage but a global rating of level 4 will only be consistently achieved after a number of procedures have been undertaken. Repeated assessment should show evidence of progression towards this level. The number of procedures required to reliably achieve level 4 will vary according to the trainee and to the complexity of the procedure, so there is no current ‘indicative number’ of cases that are required for any procedure to demonstrate competence. Early feedback of areas requiring development are of value and a less than ’perfect’ PBA is to be expected when learning a new procedure.
Attainment of the top rating for a particular assessment does not mean that there is no need to undertake further assessments of that type. As learning and training tools, the assessments continue to have value, and particularly with the assessments of technical skills, it is important to confirm that competence has been maintained.
Verbal and written interpretation of the ratings enhances the validity of WBA and ensures that the trainee receives the type of constructive criticism that should result in a reduction of errors and an improvement in quality of care.
Feedback needs to be couched in a way that leads to the improvement of the trainee’s performance rather than damaging the trainee’s confidence. A useful starting point for discussion is to invite the trainee to provide their own insight into the events and their own performance. The feedback noted in the free text feedback box should include what went well, what areas need to be improved and an agreed strategy for meeting the trainee’s learning needs.
– Trainees and trainers need to be actively motivated to use WBA methods for learning and as an indicator of progression, rather than as ‘mini-exams’.
– Assessments should be spread throughout each placement with evaluations from different assessors with different patients in different settings.
– Assessments of new tasks should be undertaken as soon as possible to aid learning through constructive feedback.
– Assessments should be punctuated with reflection on feedback and further study.
– Assessments that are consistently below standard should be tackled with targeted training and additional assessments.
– Both trainers and trainees should be prepared to trigger and accept the triggering of WBAs.
– It is essential that everyone who is responsible for assessing trainees understands how to use the rating scales and how to provide constructive feedback aimed at helping trainees to learn.
An Interim Review panel, which includes one of the training programme directors, will convene and review all the evidence in trainees’ ISCP portfolio.
The Interim Reviews usually take place in December/January and are intended to pick up how trainees are progressing towards meeting the requirements of their ARCP, which will take place in the summer. Although the Interim Reviews are informal and there is no outcome issued, attendance is mandatory and we hope trainees will take note of the guidance provided.
If trainees do not progress as expected at this point in their training, the panel will inform the trainee which areas need to improve to achieve a satisfactory ARCP outcome for their current training year. If trainees do not have good evidence of competencies, including the minimum numbers of assessments, the school will investigate the situation with the local surgical faculty.
In building portfolios, it is vital trainees make every effort to use the assessments routinely and add a substantial number of them on a regular and on-going basis.
All trainees will be required to attend a face to face meeting with the panel and will be sent a date and time for this. Trainees will be required to have their ISCP portfolio up-to-date with the requirements by the time of the review.