Roel Bogie

Development and validation of an educational web-based system for endoscopic classification of laterally spreading tumors

Discussion This is the first study to test the validity of the endoscopic Kudo classification of LSTs in a group of colonoscopy experts and endoscopy fellows. There is moderate to substantial IOA in applying the endoscopic Kudo classification and in classifying LSTs as granular vs non-granular subtype among international experts. Because the risk of containing SMI in endoscopic subtypes of LST varies considerably, correct classification of LSTs is an essential step in selecting the preferred therapy to optimize the patient outcome. Moderate to substantial agreement was found between experts on both full endoscopic Kudo classification and on classifying LSTs into granular and non-granular. However, the Gwet’s AC1 value was higher for the granular/non-granular classification than for the endoscopic Kudo classification (0.75, 95% CI: 0.66 – 0.83 vs 0.62, 95% CI: 0.55 – 0.69). This may indicate that it is easier to distinguish granular LSTs from non-granular LSTs than recognizing the presence/absence of a dominant nodule or depression within granular and non-granular LSTs. The highest proportion of discordant pairs was observed between flat and pseudo-depressed non-granular LSTs (10.7%). Thus, experts had most disagreement in recognizing pseudo-depressions within non-granular LSTs. Endoscopy trainees pursued the specifically designed e-learning on endoscopic Kudo classification. Their IOA improved significantly with kappa values approaching those of the experts. Such training helps to implement the Kudo classification into clinical practice. Although training significantly improved the Gwet’s AC1, it remained in the moderate range. As with the experts, classifying LSTs into granular and non-granular resulted in higher IOA: substantial agreement before training with almost perfect agreement after training. In a significant proportion of cases, discordant pairs between LST-G-H and LST-NG-FE were found (9.6% for experts and 9.2% for fellows [before training]). Training effectively improved granular recognition, since only 5.2% of all observations were discordant pairs between LST-G-H and LST-NG-FE after training. The highest rate of discordant pairs was seen between flat elevated and pseudo-depressed non-granular LSTs, both before and after training, confirming the observation in the expert group. Furthermore, the proportion of discordant pairs between LST-G-H and LST-G-NM decreased after training, indicating that training made the recognition of a dominant nodule easier. For each individual case concordance of answers between endoscopy fellows and experts as group could be interpreted as diagnostic accuracy, assuming that the opinion of the majority of experts is the reference. Initially, 68.1% of the cases scored by endoscopy fellows were correctly classified according to the overall expert’s decision. This increased to 76.5% after training, but still almost a quarter of all trainee ratings were misclassified according to the experts' rating. Again, after recoding into only granular/non-granular LSTs, the concordance increased (87.3% pre-training and 93.1% post-training). It is obvious that not only within but also between the rater groups (experts and endoscopy fellows), the largest disagreement exists in determining whether pseudo depression is present. A Korean multicenter study examined the IOA of the full endoscopic Kudo classification of LSTs in experts and trainees. 11 Without special LST training, the Fleiss kappa coefficient for IOA was 0.73, comparable to the present study, with the highest agreement in the LST-G-NM category. The untrained fellows had a significantly lower IOA than experts (kappa coefficient: 0.55 vs 0.73). In the present study we showed that training increased the level of IOA among trainees almost reaching that of the experts. The higher overall Fleiss kappa coefficient in the Korean study could have been the result of including raters from the same academic center or centers from the same geographic region. Previous exchanges and communication between centers could have improved the IOA in

2

31