alexa Pilot testing of an Assessment tool of Otolaryngology Residents Performing Thyroid Surgery by Using the Global Rating of Thyroid Surgical Skills | Open Access Journals
ISSN: 2167-7948
Journal of Thyroid Disorders & Therapy
Make the best use of Scientific Research and information from our 700+ peer reviewed, Open Access Journals that operates with the help of 50,000+ Editorial Board Members and esteemed reviewers and 1000+ Scientific associations in Medical, Clinical, Pharmaceutical, Engineering, Technology and Management Fields.
Meet Inspiring Speakers and Experts at our 3000+ Global Conferenceseries Events with over 600+ Conferences, 1200+ Symposiums and 1200+ Workshops on
Medical, Pharma, Engineering, Science, Technology and Business

Pilot testing of an Assessment tool of Otolaryngology Residents Performing Thyroid Surgery by Using the Global Rating of Thyroid Surgical Skills

Khalid H AL-Qahtani*
Deparment of Otolaryngology-Head and Neck Surgery, College of Medicine, King Saud University, KSA
Corresponding Author : Khalid H AL-Qahtani
Deparment of Otolaryngology-Head and Neck Surgery
College of Medicine, King Saud University, KSA
Tel: 966-11- 4775524
E-mail:
kqresearch@hotmail.com
Received: July 4, 2015; Accepted: July 23, 2015; Published: July 30, 2015
Citation: AL-Qahtani (2015) Pilot testing of an Assessment tool of Otolaryngology Residents Performing Thyroid Surgery by Using the Global Rating of Thyroid Surgical Skills. Thyroid Disorders Ther 4:189. doi:10.4172/2167-7948.1000189
Copyright: © 2015 AL-Qahtani HK, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Related article at Pubmed, Scholar Google

Visit for more related articles at Journal of Thyroid Disorders & Therapy

Abstract

Background: There is a pressing need to evaluate the performance of surgical resident in the operation theater objectively by using a structured assessment tool. This will eliminate the possibility of bias which might be a concern when the assessment is subjective. An Objective Structured Assessment of Technical Skills (OSTAT) based tool was tested using Global Rating of Thyroid Surgical Skills (GRTSS) in terms of feasibility, reliability and construct validly in evaluating otolaryngology head and neck surgery residents at various postgraduate years. Objectives: A prospective observational study using a global rating evaluation form developed in conjunction with the OSATS to assess the thyroid surgical skills of residents. The study was conducted in King Abdul Aziz University hospital, Riyadh, KSA. A total of 23 thyroid surgeries performed by 7 residents at three different levels were assessed. The participants’ scores were calculated as percentages and were entered using SPSS (Statistical Package for the Social Sciences, version 17.0) for reliability and construct validity measures. Methods: Construct validity was established by comparing senior and junior years (P<0.001, mean difference: 11.375, 95% confidence interval 6.25-16.49) and by comparing same residents in two different levels (P<0.005, mean difference: 18.8, 95% confidence interval 7.45-30.14). Proved reliability with a value of Cronbach alpha of 0.707. Conclusions: This pilot study has shown this tool to be a valid, feasible, and reliable instrument to assess the thyroidectomy surgical skills. Potential applications

Abstract

Background: There is a pressing need to evaluate the performance of surgical resident in the operation theater objectively by using a structured assessment tool. This will eliminate the possibility of bias which might be a concern when the assessment is subjective. An Objective Structured Assessment of Technical Skills (OSTAT) based tool was tested using Global Rating of Thyroid Surgical Skills (GRTSS) in terms of feasibility, reliability and construct validly in evaluating otolaryngology head and neck surgery residents at various postgraduate years.

Objectives: A prospective observational study using a global rating evaluation form developed in conjunction with the OSATS to assess the thyroid surgical skills of residents. The study was conducted in King Abdul Aziz University hospital, Riyadh, KSA. A total of 23 thyroid surgeries performed by 7 residents at three different levels were assessed. The participants’ scores were calculated as percentages and were entered using SPSS (Statistical Package for the Social Sciences, version 17.0) for reliability and construct validity measures.

Methods: Construct validity was established by comparing senior and junior years (P<0.001, mean difference: 11.375, 95% confidence interval 6.25-16.49) and by comparing same residents in two different levels (P<0.005, mean difference: 18.8, 95% confidence interval 7.45-30.14). Proved reliability with a value of Cronbach alpha of 0.707.

Conclusions: This pilot study has shown this tool to be a valid, feasible, and reliable instrument to assess the thyroidectomy surgical skills. Potential applications will be useful in tracking resident development throughout postgraduate training and certifying operative skills.
Keywords

Pilot study; Assessment tool; Thyroid surgical skills (GRTSS); Thyroid surgery; Resident
Introduction

Objective assessment of surgical skills of trainees during a residency program is crucial in the education and development of the surgical skills of trainees. Methods of assessing the performance of the residents in the operating theatre is vital in order to achieve competency in technical skills and acquisition of surgical expertise ultimately ensuring best optimal patient safety [1]. Efforts have been made to design assessment tools aiming to evaluate surgical skills in the operating theater that meet the standards of feasibility, reliability and validity. The Objective Structured Assessment of Technical Skills (OSATS) was developed by Winckel et al. [1] to conduct a formal assessment of surgical resident’s technical skill, which was later modified by Martin et al. [2].

Numerous researches in the surgical literature has been able to develop OSATS-based assessment tools with proven feasibility, reliability and validity in different surgical disciplines [3-5]. In otolaryngology head and neck surgery, such objective tools have been successfully examined as being reliable and valid in different procedures such as mastoidectomy, sinus endoscopic surgery and laryngoscopy [6-8].

Thyroid surgery can be considered as an ideal representation of head and neck procedures in which surgical residents need to have integrated knowledge of the challenging anatomy and requires development of meticulous surgical technical skills, hence the demand for a structured observational assessment is undeniable [9]. Intending to address this issue, an OSATS-based instrument using Global Rating of Thyroid Surgical Skills (GRTSS) has been designed in this pilot study to assess the resident’s expertise in technique for thyroid surgery. The purpose of this pilot study was to test this tool’s feasibility, reliability and construct validly in evaluating otolaryngology head and neck surgery residents at various postgraduate levels.
Materials and Methods

After obtaining an approval from the ethical committee to carry out this prospective study, consent was obtained from residents at various levels of training and enrolled. The staff assessors were informed about the study and agreement was reached about their participation.

The design was a prospective, observational study within the operating theatre of a teaching hospital. Trainees were directly observed performing thyroidectomy.

A 2-page evaluation form was developed in conjunction with the OSATS evaluation form that was originally developed by Winckle et al and modified by Martin et al. [2,3] (Table 1) to assess the surgical skills of residents. A one to five (five = excellent) scale was used for evaluation.

The (GRTSS) evaluation instrument (Table 2) was designed with input from academic otolaryngologist, fellowship trained head and neck surgeons and experts in medical education. Their comments and suggestions were incorporated, establishing face and content validity. Finally 16 items of task performance were selected with a five-point Likert scale linked to specific descriptors. The number five in the scale was equivalent to the performance of a resident who can perform that step independently and safely.

The participants’ scores were calculated as percentages and were entered using SPSS (Statistical Package for the Social Sciences, version 17.0) for reliability and construct validity measures.
Results

A total of 23 thyroid surgeries were performed by 7 residents at three different levels of postgraduate years; R3, R4 and R5. They were assessed by a single assessor using two different assessment tools the Global Rating Score (GRS) and Global Rating of Thyroid Surgical Skills (GRTSS) displayed in Tables 1 and 2 respectively.

The assessors reported that completing the assessment forms were convenient and not time consuming which is a major issue in an operation theater setting. The tools were also described as being simple, brief and to the point by the assessors. Based on these observations the instrument can be considered as feasible.

The interterm reliability was determined by the assessment tool’s internal consistency. It was evaluated with Cornbach’s α, which proved the tool’s high reliability, 0.707.

It has proven to be construct-valid as senior trainees achieved higher scores than junior trainees. This was measured comparing means using independent t sample test (P<0.001, mean difference: 11.375, 95% confidence interval 6.25-16.49). The GRS also showed statistical significance (P<0.005 mean difference: 6.958, 95% confidence interval 2.787-11.129).

Furthermore, to provide evidence of construct validity the a set of residents performing the surgeries were assessed by the same assessor in two separate training years, first in year 4 and next in year 5. The scores was compared between the trainees attending senior and junior years. Notable improvement was observed in the scores of the same trainees in senior year (Figure 1), proving the construct validity of the instrument. This was measured comparing means using independent t sample test (P<0.005, mean difference: 18.8, 95% confidence interval 7.45-30.14). However, the in this set, GRS did not show statistical significance (P<1.80 mean difference: 12.2 95% confidence interval 6.96-31.36).
Discussion

The emphasis on work-place assessment methods has been growing as measuring a trainee’s knowledge, skills, judgment or professional behavior is of determinative values in the educational or training programs [10]. Standardized methods of assessing the surgical skills of trainees in the operating theatre require valid and reliable instruments to ensure that graduates’ performance is in compliance with the accepted professional demands [10]. To address the growing need for a structured assessment tool to properly evaluate the operative skills of otolaryngology trainees/residents performing thyroid surgery this assessment tool was developed, which proved to be valid and reliable as well as acceptable to assessors. The feasibility and cost-effectiveness of the assessment tool developed in this study were present, as these are essential components for an assessment method to be convenient and for proper implementation of it in a training program.

Considering the reliability, the internal consistency was found to be significant when compared to the known global rating scale that was used as a second tool in this study.

Since measuring the progress of trainees over time is crucial in certifying competence, this instrument was able to demonstrate achievement of higher score with advancement in training years. It was also able to capture progress within the same participant after gaining more experience and knowledge with subsequent evaluations.

Limitations to this study include the small sample size, however, increase in the number of participants would be considered to confirm findings in future studies as this is a preliminary pilot one. Another limitation which too will be addressed in the subsequent larger scale study is that inter-rater agreement was not possible to be calculated since every participant was assessed by a single assessor.

The otolaryngology residents were directly observed performing thyroid surgeries on patients in presence of the assessors who were consultants, in contrast to other studies that were done at other different settings using bench models or live anaesthetized animals [2].

This tool’s components are broadly similar to those developed by Diaz Voss Varela et al. [11], however, this instrument has longer and separate items that would play an important role in accurate evaluation of operative skills of the residents. This study’s results were consistent with those developed by Diaz Voss Varela et al. in terms of feasibility, reliability and construct validity [11].

The experience of otolaryngology head and neck surgeons in developing assessment tools for evaluating technical skill and knowledge of trainees performing various procedures have been successful as these tools were found to have proven statistical significance [7-9]. This ensures that such validated instruments can be adopted with good educational impact in otolaryngology practice.
Conclusion

This instrument shows promising results to be a reliable, valid and feasible assessment method for testing the surgical skills of the residents within the operation theater, performing thyroidectomy. It provides detailed evaluation with global considerations to the decisive and crucial steps in the surgical procedures that requires competence of the residents. Such instrument can contribute to the educational development in the surgical field as it gives insight and feedback about the competency level of trainees.
References












Tables and Figures at a glance

Table icon Table icon
Table 1 Table 2

 

Figures at a glance

Figure
Figure 1
Select your language of interest to view the total content in your interested language
Post your comment

Share This Article

Relevant Topics

Recommended Conferences

Article Usage

  • Total views: 11488
  • [From(publication date):
    August-2015 - May 25, 2017]
  • Breakdown by view type
  • HTML page views : 7757
  • PDF downloads :3731
 
 

Post your comment

captcha   Reload  Can't read the image? click here to refresh

Peer Reviewed Journals
 
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals
International Conferences 2017-18
 
Meet Inspiring Speakers and Experts at our 3000+ Global Annual Meetings

Contact Us

Agri, Food, Aqua and Veterinary Science Journals

Dr. Krish

agrifoodaquavet@omicsonline.com

1-702-714-7001 Extn: 9040

Clinical and Biochemistry Journals

Datta A

clinical_biochem@omicsonline.com

1-702-714-7001Extn: 9037

Business & Management Journals

Ronald

business@omicsonline.com

1-702-714-7001Extn: 9042

Chemical Engineering and Chemistry Journals

Gabriel Shaw

chemicaleng_chemistry@omicsonline.com

1-702-714-7001 Extn: 9040

Earth & Environmental Sciences

Katie Wilson

environmentalsci@omicsonline.com

1-702-714-7001Extn: 9042

Engineering Journals

James Franklin

engineering@omicsonline.com

1-702-714-7001Extn: 9042

General Science and Health care Journals

Andrea Jason

generalsci_healthcare@omicsonline.com

1-702-714-7001Extn: 9043

Genetics and Molecular Biology Journals

Anna Melissa

genetics_molbio@omicsonline.com

1-702-714-7001 Extn: 9006

Immunology & Microbiology Journals

David Gorantl

immuno_microbio@omicsonline.com

1-702-714-7001Extn: 9014

Informatics Journals

Stephanie Skinner

omics@omicsonline.com

1-702-714-7001Extn: 9039

Material Sciences Journals

Rachle Green

materialsci@omicsonline.com

1-702-714-7001Extn: 9039

Mathematics and Physics Journals

Jim Willison

mathematics_physics@omicsonline.com

1-702-714-7001 Extn: 9042

Medical Journals

Nimmi Anna

medical@omicsonline.com

1-702-714-7001 Extn: 9038

Neuroscience & Psychology Journals

Nathan T

neuro_psychology@omicsonline.com

1-702-714-7001Extn: 9041

Pharmaceutical Sciences Journals

John Behannon

pharma@omicsonline.com

1-702-714-7001Extn: 9007

Social & Political Science Journals

Steve Harry

social_politicalsci@omicsonline.com

1-702-714-7001 Extn: 9042

 
© 2008-2017 OMICS International - Open Access Publisher. Best viewed in Mozilla Firefox | Google Chrome | Above IE 7.0 version