1.Comparison between inter-rater reliability and inter-rater agreement in performance assessment.
Shih Chieh LIAO ; Elizabeth A HUNT ; Walter CHEN
Annals of the Academy of Medicine, Singapore 2010;39(8):613-618
INTRODUCTIONOver the years, performance assessment (PA) has been widely employed in medical education, Objective Structured Clinical Examination (OSCE) being an excellent example. Typically, performance assessment involves multiple raters, and therefore, consistency among the scores provided by the auditors is a precondition to ensure the accuracy of the assessment. Inter-rater agreement and inter-rater reliability are two indices that are used to ensure such scoring consistency. This research primarily examined the relationship between inter-rater agreement and inter-rater reliability.
MATERIALS AND METHODSThis study used 3 sets of simulated data that was based on raters' evaluation of student performance to examine the relationship between inter-rater agreement and inter-rater reliability.
RESULTSData set 1 had high inter-rater agreement but low inter-rater reliability, data set 2 had high inter-rater reliability but low inter-rater agreement, and data set 3 had high inter-rater agreement and high inter-rater reliability.
CONCLUSIONInter-rater agreement and inter-rater reliability can but do not necessarily coexist. The presence of one does not guarantee that of the other. Inter-rater agreement and inter-rater reliability are both important for PA. The former shows stability of scores a student receives from different raters, while the latter shows the consistence of scores across different students from different raters.
Clinical Competence ; Curriculum ; Education, Medical ; Humans ; Psychometrics ; Reproducibility of Results ; Statistics as Topic ; Statistics, Nonparametric ; Students, Medical ; Task Performance and Analysis ; Validation Studies as Topic
2.Off-Label Application of Pipeline Embolization Device for Intracranial Aneurysms
Buqing LIANG ; Walter S LESLEY ; Timothy M ROBINSON ; Wencong CHEN ; Ethan A BENARDETE ; Jason H HUANG
Neurointervention 2019;14(2):116-124
PURPOSE: The Pipeline embolization device (PED) is approved in the USA for treating giant and large aneurysms arising from the petrous to superior hypophyseal segments of the internal carotid artery in patients older than 21 years of age. This study investigates off-label PED results in a large cohort. MATERIALS AND METHODS: Retrospective, single-center review of all patients who had off-label PED surgery. RESULTS: Sixty-two aneurysms (48 patients) underwent off-label PED treatment from 2012–2017. There were 44 females and four males (age 21 to 75 years; mean/median, 54.3/55.0 years). The most common presenting symptom was headache (47/62, 75.8%). All aneurysms were in the anterior circulation. Aneurysm size ranged from 1.4 to 25.0 mm (mean/median, 7.6/6.9 mm). Fifty-two aneurysms had post-operative imaging with total/near-complete occlusion of 84.6% (44/52). Aneurysm-based operative near-term complication rate was 9.7% while there were no permanent complications. For aneurysms and headache, 86.7% improved/resolved after embo-surgery, and were four times more likely to have a better clinical outcome (resolved or improved symptoms) after surgery (odds ratio [OR], 4.333; P=0.0325). Left-sided aneurysms had a higher occlusion rate (OR, 20; P=0.0073). Hypertension (OR, 4.2; P=0.0332) and smoking (OR, 7; P=0.0155) were more prone towards aneurysm occlusion. Patients without a family history were 14 times more likely to have favorable imaging outcome (P=0.0405). There is no difference of occlusion rates between untreated and previously treated aneurysms (P=0.6894). Overall, occlusion rate decreased by 14% with an increase of aneurysm size by 1 mm (P=0.0283). CONCLUSION: For anterior circulation aneurysms, the off-label application of PED is as effective and safe as reported for on-label intracranial aneurysms.
Aneurysm
;
Carotid Artery, Internal
;
Cohort Studies
;
Female
;
Headache
;
Humans
;
Hypertension
;
Intracranial Aneurysm
;
Male
;
Retrospective Studies
;
Smoke
;
Smoking
3.Clinical skills in final-year medical students: the relationship between self-reported confidence and direct observation by faculty or residents.
Walter CHEN ; Shih-chieh LIAO ; Chon-haw TSAI ; Chiu-ching HUANG ; Cheng-chieh LIN ; Chang-hai TSAI
Annals of the Academy of Medicine, Singapore 2008;37(1):3-8
INTRODUCTIONMany students, while performing clinical skills such as medical interviewing/ communication, physical examination, and procedural tasks, have never been observed by faculty members or residents. This study aimed to explore the relationships between final-year medical students' self-reported confidence and the frequency of direct observation by faculty member or resident while conducting these clinical skills.
MATERIALS AND METHODSMedical students at China Medical University in Taiwan participated in the survey. Before graduating, they were asked to answer a questionnaire about (1) their confidence in performing 17 clinical skills including medical interviewing/communication, physical examination, and procedural tasks, and (2) the number of times they had been directly observed by faculty members or residents during student-patient encounters.
RESULTSMany students reported never having been observed by a faculty member while they performed history taking/communication (46% to 84%), physical examination (36% to 42%), or procedural tasks (41% to 81%). It was found that residents had observed the students more frequently than the faculty members. The correlations between self-reported confidence and the corresponded direct observation were small to medium but significant. However, no difference was found between observation by a faculty member and by a resident.
CONCLUSIONSThis study confirmed that many medical students have not been directly observed in clinical training; and that those who were observed more often, expressed more self-reported confidence. Some assessment measures, which focus on direct observation and feedback during student-patient encounters, may improve the students' confidence.
Adult ; Clinical Competence ; standards ; Data Collection ; Female ; Humans ; Internship and Residency ; Male ; Observation ; Self Efficacy ; Students, Medical ; Taiwan
4.Potential therapeutic effects of dipyridamole in the severely ill patients with COVID-19.
Xiaoyan LIU ; Zhe LI ; Shuai LIU ; Jing SUN ; Zhanghua CHEN ; Min JIANG ; Qingling ZHANG ; Yinghua WEI ; Xin WANG ; Yi-You HUANG ; Yinyi SHI ; Yanhui XU ; Huifang XIAN ; Fan BAI ; Changxing OU ; Bei XIONG ; Andrew M LEW ; Jun CUI ; Rongli FANG ; Hui HUANG ; Jincun ZHAO ; Xuechuan HONG ; Yuxia ZHANG ; Fuling ZHOU ; Hai-Bin LUO
Acta Pharmaceutica Sinica B 2020;10(7):1205-1215
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection can cause acute respiratory distress syndrome, hypercoagulability, hypertension, and multiorgan dysfunction. Effective antivirals with safe clinical profile are urgently needed to improve the overall prognosis. In an analysis of a randomly collected cohort of 124 patients with COVID-19, we found that hypercoagulability as indicated by elevated concentrations of D-dimers was associated with disease severity. By virtual screening of a U.S. FDA approved drug library, we identified an anticoagulation agent dipyridamole (DIP) , which suppressed SARS-CoV-2 replication . In a proof-of-concept trial involving 31 patients with COVID-19, DIP supplementation was associated with significantly decreased concentrations of D-dimers ( < 0.05), increased lymphocyte and platelet recovery in the circulation, and markedly improved clinical outcomes in comparison to the control patients. In particular, all 8 of the DIP-treated severely ill patients showed remarkable improvement: 7 patients (87.5%) achieved clinical cure and were discharged from the hospitals while the remaining 1 patient (12.5%) was in clinical remission.