Observations of surgeons as a tool to assess competency

Observations of surgeons as a tool to assess competency

Posted by Annelies Querner-Verkerk on Tue 15 Jan. 2013 - 2 minute read

Medical doctors receive extensive education and training before and during their time spent working at a hospital or clinic. There are many training sessions organized by groups of medical specialists, hospitals, and professional societies worldwide that doctors can choose from to continue improving their communication and technical skills.

How can we consistently evaluate performance?

Skills and knowledge should be evaluated on a regular basis, shouldn’t they? Many researchers are receiving funding to investigate how skills are best taught and practiced.

Also, effective communication and positive team interactions are becoming increasingly recognized as crucial components for success in the workplace. Studies are investigating the best way for a physician to speak to a patient while simultaneously obtaining the appropriate medical information and putting the patient at ease.

Below you can find a summary of an observational research study conducted by Miskovic et al. (2012) about a new method used to assess the competency level of surgical specialists in advanced laparoscopic surgery.

“Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level”

Recently, Miskovic et al. (2012) collected 32 video recordings of laparoscopic colorectal resections. Laparoscopic surgery is a modern method in which small incisions are made. It is meant to minimize post-operative pain by using smaller incisions and speed up recovery times. The surgeon uses small tools to interact with tissue. Experts and delegates of the National Training Program in England performed all surgeries.

In total, Miskovic et al. evaluated performances of 4 experts and 27 delegates of which 17 passed the National Training Program in England. With this material, Miskovic et al. were able to evaluate a new method called OCHRA that has been developed to detect errors and near misses during surgeries.  

Video analysis is at the core of this method. Miskovic et al. used The Observer XT to code events and calculate frequency and duration. The full coding scheme can be found in Surgical Endoscopy. All videos were analyzed by experts (expert reviews) and by observers using the predefined coding scheme: time spent on Dissecting (D) and on Exposing (E) tissue were measured. And, all errors were marked as point events. If an error led to a direct consequence (i.e. when the patient started bleeding), this was also recorded.

After accurately coding the videos, the complete error count was extracted from The Observer XT for further analysis in SPSS. 









FREE TRIAL: Try The Observer XT yourself!

Request a free trial and see for yourself how easy behavioral research can be!

  • Work faster
  • Reduce costs
  • Get better data


Expert review and behavioral analysis

In this study, two methods were used: expert review and behavioral analysis. Miskovic et al. explain that novel methods are required to further rationalize and objectify the process of measuring competence. Individual assessments of operating videos can easily be combined with detailed behavioral analysis to allow for more accurate assessments. 

Concluding

The researchers concluded that OCHRA is a valid method for assessing performance of surgical specialists. The video analysis delivered the researchers with proof that experts have a higher D/E ratio, in other words work more efficient, in comparison to delegates. Note that it is not the operating time that determined the outcome. It is the combination between time spent on for example cutting tissue and exposing tissue time (i.e. grabbing a different piece of tissue).

OCHRA can now be integrated into clinical training programs. Miskovic et al. emphasize the importance of assessing the competence of surgeons at this level by evaluating real operations in a real environment.

Reference

Miskovic, D.; Ni, M.; Wyles, S.M.; Parvaiz, A.; Hanna, G.B. (2012). Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level. Surgical Endoscopy26,796-803.

Don't miss out on the latest blog posts
Share this post
Topics
Learn
more
Relevant Blogs
operating-room-layout

Operating room layout: impact on work patters and flow disruptions

With use of prerecorded videos of surgeries, researchers examined the impact of the layout of an OR on work patterns and flow disruptions of a circulating nurse.
evaluating-ergonomics-in-healthcare

Evaluating ergonomics in healthcare – paramedics

The chance that a paramedic suffers an injury is 3x higher than other occupations. Because of the global health worker shortage, we need to make sure these professionals are taken care of.
imsh2023

5 reasons to attend IMSH 2023, a.k.a. the medsim conference

The Society for Simulation in Healthcare (SSH) organizes the International Meeting on Simulation in Healthcare (IMSH), which is the world’s largest scientific conference.